Outcome focus, Benefits, and Performance Management

Successful delivery of service improvement depends on knowing what is actually happening, and making decisions using evidence (often termed Performance Management or Performance Improvement). All too often decisions are taken based on assumptions or suppositions, and audits measure things that nobody thinks are important.
My rapid assessment process has been used early in a number of service transformation projects, both to motivate and engage staff, and to assess the direction and progress of a project and any changes needed, early enough to make decisions which make a difference. Rapid Assessment leads to Rapid Decisions

many audit and performance tools suffer:
they aren't seen as relevant - the things they measure don't mean anything either to the stakeholders or to whether the project is actually delivering benefits (milestones based on time rather than achievement, for example)
it takes too much effort to gather the information - the measures are too complex, too involved, or not clearly defined
the results are too late to make a difference
We typically need results in time to make a decision based on those results, and audit and measurement often have such a time delay built in that the decision-point is passed, and to change direction at the point of decision will cause a lot of expensive undoing.
Minney.org has developed processes which we work through with the stakeholders to:
Picture from Canada's library - shoal of baraccudadefine measures which are common sense and relevant
measure and report in real time so the people on the ground can see what is going on, and highlight potential changes of direction for appropriate decisions
focus on outcomes and what the project is trying to achieve. This also means that as the organisation's priorities change, so individual projects can change their own target outcomes - a shoal of fish changing direction instantaneously and without waves, rather than a massive whale.
see the pages below for more information

Benefits Realisation - The Benefits Design Process

A colleague asked me over a coffee “how do you go about identifying, defining and quantifying benefits? How do you create a benefits realisation plan?”. I think hidden behind the question was a number of thoughts:

  1. Were you just bluffing?
  2. Can we really realise these benefits if we only follow the project plan we ourselves prepared (with coaching and support, of course)?
  3. Should we aim to only realise a % of the benefits, in other words is the Benefits Realisation Plan a work of optimism to get the investment?
I’m always delighted to share my methods. For those who can’t get an outside consultant with expertise in this area, this page helps so you can prepare to realise the benefits of new projects and care for the populations you serve and steward public resource. For those who are reading about me with an intention to hire, it means you can sanity check my process.
NOTE this description could just as easily have sat in Benefits Realisation – Good Projects in Public Sector section. Words underlined are due to have further information added though it isn’t complete just yet.
 

Mature

My process has matured over the years, and I’m delighted to say has stood the test of time – even the very rapid assessment process which can be run within months of the start of delivery can yield an estimate of future benefits, and when we’ve followed up the initial rapid assessment over a period of years, the additional data bears out the initial findings (though of course with a lot more statistical confidence).
The process is based on the confluence of a number of different themes: the balanced scorecard, Bernice McCarthy’s 4-mat, Cranfield benefits planning, OGC MSP and OGC Prince2 updated for 2009.
For ease of representation I recommend using a [Benefits Dependency Network] template. I’ll expand each area as time goes on
 

Begin with the end in mind

  1. Every investment has [strategic objectives] (one or sometimes a few). These are the key, and your benefits will contribute to these. Why are you doing this project?
  2. What do you need to be (what will the organisation or project team look like, be like, be doing) in order to achieve your objectives?
  3. How will you achieve this?
  4. Between 2 and 3 are the benefits, the things that you can measure and that everyone agrees represent making a difference
  5. Sanity-check the benefits you’ve identified in step 4 against a balanced scorecard
 
Process
People
External
Clinical or Quality Outcomes
Patient or User experience
Internal
Effective use of resources
Impact on Staff and ability to retain and recruit
 
  1. [Sanity check the Benefits Dependency Network] (from the left, HOW, Benefits, WHAT, WHY) checking for
a.       benefits not tied to the strategic objective(s)
b.      double commitments (benefits with not enough actions to realise them)
c.       actions that don’t make a contribution
d.      etc
  1. for each benefit, with the frontline staff who do the work and have the most invested in achieving a successful outcome (you can also involve other stakeholders eg service users)
a.       define [how you will measure progress and achievement] (it doesn’t matter whether the frontline staff are actually doing the recording and measurement themselves or whether it’s done through another function; it is important that they decide what shows whether the service is working or not)
b.      decide how the measures will be reported – remember prepare your reports for staff and service users as board members are people too
c.       “people do what you inspect, not what you expect” . Be mindful of what happened to Shroedinger’s cat (the process of measuring affects the system being measured) and [be careful what you measure]
This process can be applied to individual services, to workstreams, and to whole health and care economies. The advantage of working this way across a larger group of individual services is that you can examine the contribution each service makes to the overall needs of the community.
At this point I must emphasise that there’s so much work needed caring for service users, nobody is going to lose their job. Far better to identify those services that don’t contribute and redeploy people, than to wake up 2 years’ from now and say “so what I’ve been doing for the last 24 months hasn’t helped anybody?” – recipe for suicide.
 
I look forward to working with you to implement this, engaging front-line and support staff in the strategic and tactical needs of the health and care of the population and ensuring we do our own management instead of needing it forced upon us.

 

Corporate Social Responsibility - making a difference for yourself and the world

Keywords:

CSR is widely criticised as an advertising tool, rather than a way to make a real difference.  But is it?

CSR (Corporate Social Responsibility) is doing something for the world, the nation and the people around you which makes it better.  It can be as little as complying with the spirit of the law, ethical standards and business norms, or it can embrace the activities of the company's staff, encouraging them to contribute in more ways.

Why do companies do it?

CSR is basically about giving a little bit back.

There are plenty of ways to "avoid" paying tax, but Warren Buffet champions a movement to pay what he owes, and other wealthy people are following suit.  Some of the companies most criticised for environmental damage and third world exploitation are now publishing how much they pay in tax, contribute to local economies, the jobs they create.

So, once you've paid your tax, you can give to your favourite charity.  This isn't very personal - gaining money, then passing it on.  Many organisations want to get involved, to get their hands dirty, and do something themselves.  Staff can teach adults with literacy difficulties or work as teaching assistants, dig gardens or convert rubble into vegetable allotments, redecorate.  Companies can fund parks or recreation.

Is there an advantage to be gained?

Yes!  Most companies that engage with their local community get good publicity. The Prince of Wales-sponsored group Business in the Community (BitC) started around 1980s with the motto "Healthy backstreets equal healthy high streets", and commercial interests were recognised.

Good publicity can mean more sales.  In one situation we observed that public procurement was prepared to pay up to 15% more for a local social enterprise to provide a service, than for a national for-profit organisation (no names to protect).  It certainly applies to consumers - a great many people use the excuse that it "gives money to charity" when they buy their lottery ticket, and there's every reason to suggest they will apply the same sentiment to their buying decisions- buy from companies with the feel-good factor.
But sales aren't everything.  Staff may be the most expensive resource, and they are certainly the biggest opportunity to create success.  A good name can improve recruitment and retention.  Programmes that involve staff help staff work together, each contributing skills that they wouldn't bring to work.  The invisible boss, working with you digging rubble and planting cabbages, suddenly becomes someone you can relate to.  People you trust = loyalty, and loyalty means a lot more than just the cost of recruitment and retention, it means an honesty and trustworthiness that may be impossible to get in any other way.
 

Does it mean less profit?

This is often the most important question that companies ask.  Will it cost me money, and will the investors object?

It doesn't have to.  I've described at the simplest and most immediate level the ways that good CSR can help a company, usually to a much greater extent than the money put into the scheme.  In most cases, the schemes don't cost much in monetary terms, they take time and energy.

An earlier study found that firms that put lots of emphasis on environmental and community contribution (such as Innocent) can make more profit.  Firms with strong ethics often have strong staff attitudes, which leads to great success.  Firms all out to make a profit, will also do so.  It's the firms in the middle, which want to make a profit but think they ought to do something about CSR, that tend to fritter away their advantage.

How you can introduce it into your company, and what you have to think about.

First of all, decide whether you're going to do it properly, or just dabble.  If you are going to dabble, then you will lose with your left hand everything you gained with your right.  A scheme by Nestle to give vouchers for sports-related activities, when you bought foods by them, was heavily criticised.

Pick activities where you have a strong interest.  If you want to gain a reputation for integrity, pay your taxes in each country and then tell everyone about it.  If you want to overcome a reputation for environmental damage, do environmental good works.

But don't be quite as cynical as that.  In every country, there are people who are deeply passionate about one thing or another.  Let them put forward their passion and support them to get others involved.

How do you make it work for you? That's where you need some external expertise.  We specialise in measuring things that are difficult to measure, like the contribution to social good.  To keep people's interests, you need to show that the work people are doing is making a difference.

If you nurse all day, you can see the difference you've made (actually this is not true - many nurses get jaded and need to be reminded of how much difference they make).  If you contribute 2 hrs per week or 2hrs per month, you quickly think that you aren't making any difference and stop contributing.

If you as an organisation are helping street kids, then the changes are long term and it is difficult to issue a press release beyond "this is what we want to do".  We can provide the measurement and reporting, using robust methodologies, which give you something worth reporting. Worth reporting to your board and non-execs.  Worth reporting to your customers.  Worth reporting to staff and investors, to potential staff, worth doing because this contribution is also a contribution to your bottom line!

See also relevant Book Review Impact Investing: Transforming How We Make Money While Making a Difference

Creating Organisational Capacity for Benefits Management

APM_SIG_Thought_2.JPGAPM (Association for Project Management) released its second Thought Leadership Guide on Benefits Management.
It's entitled Delivering benefits from investment in change - creating organisational capability, and it describes how to make the next step from Strategy into capability.

Association for Project Management's Benefits Management Special Interest Group (APM BM SIG) includes some of the (English Speaking) world's leading experts on the subject, including Steve Jenner (Director at the Criminal Justice Intervention Team), John Thorpe of Sigma Solutions, Gerard Bradley and Peter Glynne.

Hugo Minney is also on the Committee and is authoring some of the guides.  This second guide follows on from Delivering benefits from investments in change - Winning hearts and minds

The Association for Project Management is an ideal platform for Benefits Management.  The APM was one of the first organisations to codify the concept of Portfolio Management, of aligning all of the change within an organisation so that it contributes to the organisation's strategic aims. Realising the strategic aims of the organisation, by definition, means realising the benefits that the organisation seeks.  So Benefits Management links closely into Project, Programme and Portfolio Management (P3) and is the ​raison d'etre​ of any organisation's change programmes.

But it isn't all plain sailing.  "Winning Hearts and Minds" recognises the intense political issues.  "Creating Organisational Capability" takes this a stage further, how you identify the right people, the people who will infect the organisation with a desire to deliver benefits.

There are two more thought leadership guides in this series.  The Benefits Management SIG has also produced one or two industry-specific guides.  Please have a look!

Innovation - how to do it

Guy KawasakiInnovation is a virtually inexhaustible resource - it doesn't run out, in fact the more you use it, like a muscle, the more capable it is.

So don't hold back on delivering the very best.

Guy Kawasaki (author and venture capitalist, and one-time chief evangelist at Apple Inc), one of the leading thinkers on Innovation, talks about the four qualities of Innovation that he believes every product needs

This is a flash video so I hope it plays on your device.  If not, the URL to the page is here: http://www.zdnet.com/videos/events/guy-kawasaki-what-makes-innovation/31...

This video is from June 2009, though it is still good.

D - Deep

I - Intelligent

C - Cheap

E - Elegant

At TSRC we are continuously developing our skills and the contribution we can make.

What sort of Project Manager are you? Lessons from 1066

Bayeaux Tapestry - DetailHow is a project like a war?  No, not because everyone is fighting and the only winners are the vultures - there is a lot more depth to it, and Stephen Carver of Cranfield Business School proceeded to illustrate this (complete with actors in full regalia, re-enacting the Battle of Hastings at Battle​ in a small hotel room) in a most memorable way at APM's North East Branch meeting on Tuesday 25 Oct.

Can you remember a project which failed because of politics?  Most of the politics related to things that had happened years before: old friendships and loyalties; old arguments and petty revenges?  How what you found out (you did do a Stakeholder Mapping Exercise, didn’t you?) turned out to be wrong?  Prof Carver took us back around 1000 years to the genealogy of the Kings of England from late 10th Century, and showed that there were three people who felt they had a claim to the throne:

  • Harald Hardrada of Norway, who inherited a “gentlemen’s agreement” with King Cnut (agreed in a bar in Ipswich?) that he would succeed the English
  • throne
  • Harold
  • Godwinson, son of King Edward the Confessor’s right hand man, the person chosen by the Angles and Saxons in
  • England
  • William
  • the Bastard, Duke of Normandy, who was a childhood friend of Edward the Confessor and was determined to inherit the kingdom of England and add it to Normandy; he also had the documentation to prove it

These three had very different characters, and it is their characters that determine what happens next.

Hardrada was of the Nike school - Just Do It.  He didn’t really believe in planning; he was big and bold, and he succeeded because he did it before anybody else had started.  Hardrada was lucky - though of course he made his own luck.  Do you know Project Managers who work like this?

William was a meticulous planner.  He planned everything, and had a Plan B and a Plan C (and a Plan D, when it came to the famous Battle), and he wasn’t afraid to get into the detail - according to Prof Carver (and who am I to dispute this), he planned how much dung would come from the 3000 horses that he took to invade England, and therefore how many wheelbarrows his army would need to keep the camp clean.  He treated soldiers and workers as completely expendable, but he didn’t lose a single soldier to disease.  He did have some emotion though - he was a control freak.  Carver showed the picture from the Bayeaux Tapestry of William laying down the law.

Harold was a people person. He supported his men, and they loved him.  One of the illustrations on the Bayeaux Tapestry (an embroidery, Carver informed us) is of William and Harold marching their army across quicksand.  William was ready to carry on, replacing the men who sank.  Harold went back and carried them out himself.  How do you treat your staff, especially your contract staff?

Harold had the loyalty and commitment of the English army. He was on the South Coast waiting to repulse William’s invasion, and Hardrada (tipped off by Harold's traitor youngest brother, Tostig) siezed his chance and invaded in the North.  The Vikings arrived, slaughtered and pillaged, and started a week long party.  It would take 2 weeks for Harold’s army from the South to reach them, so they had plenty of time.  But Harold covered the 185 miles in 4 days (think about it - he had VERY loyal staff - fully armed for battle) and (exhausted as they must have been) they made short work of the Vikings.  Meanwhile William had invaded with an enormous Holy Army (William had persuaded the Pope to declare this a Holy War and excommunicate Harold) and had plenty of time to dig in, and made a point of harrowing the locals so that Harold would get angry (because of his love for the English people).

William Duke of Normandy - it's my way or the highway (well, execution)Harold was back down in no time - remember this is an army that has marched 370 miles in 10 days and won one battle, and now they are ready to fight another - and Harold planned his battle tactics well. 

The defenders (Harold, in case you've lost track) were winning, when William did something completely out of character - he did something people-focussed (he removed his helmet - dangerous in the middle of a battle, and showed that he was still alive, which rallied his fleeing cavalry).  Then he unveiled Plan D - a hit squad to kill Harold at any cost (about 30 soldiers set out, only 3 or 4 actually reached Harold), and so won the war.

So much about projects depends on Luck.  But a lot does depend on the style of the project manager.
A Hagrada, an action man, can look great against the small challenges, but fall apart when he meets one that is his equal.
A meticulous planner, a William, will be hated by everyone around, and will have to bring overwhelming force to succeed, which tends to cost more.  But with access to overwhelming resources, William can succeed.
Harold, loved by his people and with real commitment to them, can achieve the impossible.  It's a shame he lost, but with the overwhelming resources of the whole of Europe and the backing of the pope, it was perhaps inevitable that William won. 

Can you afford overwhelming resources?  Can you afford hit squads where most of the resources are lost?  Didn't think so.  That’s why TSRC is about inspiring people to enjoy the work they do.

And another thing.  The way it is reported isn't necessarily the way it happened.

Harold, William and Hagrada were all Norsemen.  The French had been thrown out of Normandy by a succession of Norse.  Harold’s father came across with King Cnut and was Norse.  And Hardrada we know about.  So the French and the English didn’t figure at all in the Norman invasion of England in 1066 (the apparent politics wasn’t true).
The arrow in the eye may have been stitched onto the tapestry in 17th century, may not be original at all.  Certainly Harold is shown being killed by having his leg and head cut off elsewhere, very much hopping and no sign of an arrow or blood.

Last of all, watch out for the Tostig in your project, determined that it won't succeed.

The Battle of Britain - a metaphor for complex projects

Prof Stephen Carver illustrates the complexity of project management in his inimitable style, when he describes the Battle of Britain.  Project managers are like individual fighter pilots, each reacting and responding to the buffeting of circumstance.  But it is vital that they are coordinated in a programme, which itself needs to be part of a structured direction (a portfolio) to deliver the organisation's objectives.

The link to his profile is http://www.som.cranfield.ac.uk/som/p1308/People/Faculty/Academic-Faculty-Listing-A-Z/Last-Name-C/Stephen-Carver here, and the video to click on is "Complex Projects and Programmes"

link to video here

The training, skills and experience of an excellent project manager can be compared to flying an aeroplane.

In the commercial world, the environment is always changing, but the unexpected rarely occurs.  The commercial pilot has to keep monitoring whether his project is on target, and makes small adjustments to keep it so.

The airport conning tower controller has to manage multiple flights, ensuring on a minute by minute basis that they don't bump into one another, that they complement each other, and that they land and discharge passengers, fill up and take off again, mostly on time.  Similarly a programme manager doesn't get involved in the minutiae of each individual project unless the project manager turns out to be incapacitated or incompetent (same situation as when an airline pilot is incapacitated).

At the portfolio level, Air Traffic Control UK or Europe has to put in place principles that govern the flow of thousands of flights per day, constantly adjusting to changes in conditions (wind direction may mean different approaches to airports, a cloud of dust from Iceland can mean a major re-routing of all flights).  Changing economic conditions, new opportunities, loss of a key resource - all can change the conditions of the project and programme portfolio.

Air Force at War

War brings different challenges.  Your enemy doesn't tell you what they are going to do next.  In fact they positively try to surprise you.  None of this "40 minutes until I start shooting you", compared with a commercial flight "3 hours until I turn left over Birmingham".

The fighter pilot has to be able to react in an instant, both responding to his or her enemy, and supporting/ not crashing into his/her friend.  The commander has to manage multiple flights whilst watching and predicting multiple enemies.  And the people directing the war have to alter their principles for engagement, and the framework within which both programme and project manager work, to make sure it is fit for the current situation and yet takes them ulitmately to where they want to go.

How apt! Projects / programmes / portfolios in a competitive enviroment!  Enjoy the video

 

Performance Targets - Incentive or Burden?

Key performance indicators (KPI), targets by another word, seem to be here to stay.

In the commercial world, they may be self imposed: budgets, forecasts, sales targets, the expectations of the city. In services for the public good, the government creates the targets. For example in health, there are 698 targets that align with Standards For Better Health[1], and another 166 that don’t[2]. In addition to these, we have QOF, activity reporting for enhanced services, and many more.

The big question is: do they improve performance or reduce it[3-4]? Let’s explore?

Targets that get in the way

Simply reporting on all these targets appears to take 25-40% of all NHS management time. How much of your time to you personally spend reporting to Department of Health or their proxies, PCT, and SHA?

But is this all a bad thing?

Targets have decimated waiting times. The 4 hour wait in A&E; the 18 week wait for commencement of treatment; the 8 minute ambulance target; the improvement in patient care has been absolutely outstanding.

Targets enable us to do what the best do; to model how the best achieve what they achieve, whilst understanding which bits are worth copying and which bits we should leave out. Targets define minimum standards, the minimum acceptable level of service, and make sure that all patients received at least that.

Targets are, in essence, the basis of regulation, and are here to stay.

Characteristics of a successful organisation

Organisations which don’t hit targets generally fail in many areas. Sure, you can have perverse incentives, targets that you achieve, but that don’t do anyone any good. The old chestnut is “the patient died but the operation was a success”. Occasionally highly visible perverse incentives overshadow many excellent and outstanding results.
The truth is, that organisations that are capable of hitting targets, are generally also capable of achieving exemplary results in every area.

Would it be wonderful if . . .?

Wouldn’t it be great to hit targets without really trying? If your only effort was to make decisions, to steer the organisation, and the results took care of themselves? Wouldn’t it be great if the reporting was simplified, to the point where it looked after itself?
In my GP practice, we tried using our relatively low cost admin staff to populate a dashboard, so that we could make decisions. The results surprised us.

Only fairly senior staff were able to run the reports needed to populate the dashboard. Our data analyst was occupied at least half time running reports and transferring data, just to populate the dashboard. It meant that when the dashboard was presented, she hadn’t had the time or space to prepare recommendations. And we lost an important member of staff, who had plenty of other duties.
This situation was unsustainable.

We identified an automated extract tool which are listed not just the performance against target, but also the remedial actions that we would need to take, to hit that target. In our case it took this a stage further, recommending priorities. Because it ran locally, it was able to list the individual patients needing care. There are other tools that run automated extracts[5], and to make the decision making process straightforward, and they apply in different environments. [6]

Getting results

Bringing in an external like me to identify where you could use automation and identify existing tools, could release potential within your own staff delegate and empower. I won’t stay long: the point is to transfer knowledge, and let you and your staff run the show.
If you feel the directors are leading, dragging everyone else kicking and screaming into the future – then you need to get everyone pulling in same direction.

Citations

1. DH, Standards for Better Health, D. Health, Editor. 2006. p. 29pp.

2. Blunden, F., Frances Blunden on the burden of NHS bureaucracy, in Health Service Journal. 2009, HSJ.co.uk.

3. Dryburgh, A., Don't you believe it . . . forecasting works, in Management Today. 2010: London. p. 16.

4. MacBride-Stewart, S.P., R. Elton, and T. Walley, Do quality incentives change prescribing patterns in primary care? An observational study in Scotland. Fam Pract, 2008. 25(1): p. 27-32.

5. NHS IC. General Practice Extraction Service - benefits for patients. 2009; Available from: www.ic.nhs.uk/gpes.

6. DH, Improving quality in primary care, in World Class Commissioning, DH, Editor. 2009, Crown.

Competition in the NHS

The marketplaceIs competition good, or bad, for services paid for out of public funds?

A lot depends on whether the competition is fair or unfair.  For example, when you go out to buy a car, the competition is generally fair.  Most people know what to expect from a car, both tangible (how many doors, comfort, performance, fuel economy) and intangible (marque, implied warranty), and most options are available to most people in most places.

Healthcare is a little different.  People don’t necessarily know whether they need a hospital visit or whether care nearer to home will actually be better for them.  Often no-one explains (in terms they can understand) what they will get for the different choices they make, and there’s no-one to ask advice – and these decisions come at a difficult and emotional time.

So this isn’t helped when people are arguing till they are blue in the face, about competition (incidentally, I suspect that a trip to hospital is NOT necessary when you are blue in the face; perhaps it’s time to stop talking and start listening?).

In PM Tony Blair’s time, everyone assumed that competition would be fair, in the sense that all healthcare was only delivered to the best possible quality.  So people (including government, I’d like to think) got a bit of a shock when the private providers came in and all sorts of distortions and unfairnesses emerged – cherry picking the easy cases but still charging the full tariff, getting paid for activity not carried out, poor quality of care with the problems then passed on to the NHS hospitals to sort out.

Managing QualityThe arguments will toss back and forth, and like any good commentator, I say “follow the money” – look at the motives behind each of the people.  For example, Mark Britnell is widely respected and is one of the key voices saying “any competition is good competition”.  Well his consultancy benefits from advising governments to change policy (well, you wouldn’t get asked in for the next job if you just said “things are fine as they are”); from implementing policy; and from clearing up the mess and designing a new policy (“well you didn’t implement it the way we said”).  Nick Clegg needs to put clear blue water between himself and David Cameron.  PM David C needs to support the international healthcare providers who funded his election campaign.  And so on.

But at the root of it all, we need good regulation.  This means clear measurement of the things that are easy to measure (Full Consultant Episodes, Outcomes, Costs, Waits), but also of the things that are less easy to measure (complexity, staff use) and of things that emerge as needing measuring.  It means teeth – a range of sanctions from fines to restricted practice before the nuclear option of removing a contract or removing a license to act. And the regulator needs to be locally accountable.

How will we get to this?  That’s a subject for another paper.

Choice and competition in the NHS

Choices - chosing the right door

Is there such a thing as patient choice?

Healthcare is now so complicated that the patient would have a great deal of trouble making a choice. If they want to know what is the best treatment for their condition, they're pretty much entirely reliant on their GP. If they want to actually choose which hospital or specialist service provider to go to, then what are they going to rely on? Probably not the league tables.

Most likely, patients will choose the most convenient option, the hospital that has the monopoly on care near their home. Even if they consider changing, it will probably be based on rumour and PR. Is this an opportunity for organisations with big marketing budgets? You bet it is! And do public service organisations have big marketing budgets? Well, they certainly don't understand marketing like private health care providers do.  Does this make competition bad, and the introduction of private providers a disaster?  No, but there are issues which need addressing.

If patients are so reliant on their GP's opinion, what does their GP used to make a decision?

The GP has a menu of services provided under Choose and Book. There are currently discussions about whether Any Willing Provider (AWP) or Any Qualified Provider (AQP) will get onto the Choose and Book menu, and whether prices will be determined as tariff, or whether competition in pricing will also be allowed.

Competition – "a big opportunity for commercial healthcare" in the UK (Mark Britnellspeaking in the USA)

My last post pointed out how naive England is about private providers. Many are outstanding, excellent. But there's nothing in the system to prevent the bad apples coming and spoiling the whole bushel. They enjoy a different regulatory regime. This makes it easier for them to publicise the good PR, and to hush up the bad – to get to the patient, and persuade them to choose on the basis of advertising rather than on the basis of merit.

It's very difficult to introduce competition in the health service.

Gearing up to provide universal health care is expensive. Short-term contracts (three years, five years) represent too much of a risk. Investing tens of millions of pounds in the building and equipment to provide services to modern standards is a big risk without 10 or 20 years to spread the risk, but our current commissioning competence is woefully inadequate for signing long-term contracts.  Let’s look at where private healthcare is the main provider, and where the bulk of the population (well, 70%) actually use the healthcare provided.

ChoicesThe example is the USA: health care costs a whopping 15% of GDP – that's around $8000 per man, woman and child, $1 in every  $6 that circulates.  This compares with the UK: 8% of GDP, $3500. The important question, I suppose, is why are we trying to imitate them?

But if the US health service is so much more expensive, does it deliver a much better service?

This is particularly difficult to answer. We in the UK, and the US, are rich enough to kill ourselves. We can afford far too much alcohol, far too many fats and calories, we can afford to do nothing – to kill ourselves through inactivity. Life expectancy at birth in the US is around 78 years for a female; compare this with 79 in the UK, and an average across Europe of 80.5. Compare Japan's life expectancy of 83 years – on second thoughts, don't. A recent examination of pension fraud in Japan estimated that around one third of pensioners claiming pensions were in fact dead, and had been for many years. It's quite possible that life expectancy in Japan is not quite as exemplary as we've been told!  What we don’t know is whether healthcare is to blame for US having worse life expectancy than UK, or is it traffic accidents and guns.

Choice may drive up quality, or it may drive it down

Put simply, choice relies on perfect knowledge, and we definitely don't have so much of that in healthcare. It is very easy for an independent provider to distort the patient perception. Driving up quality needs a level of regulation which is not in the current health proposals.

Let’s explore quality in the next blog post.

Case Studies on Competition in the NHS

SheffieldWhen I arrived in Sheffield in 2004, my first task was to "kill" the ISTC (Independent Sector Treatment Centre) that had been "imposed" on the South Yorkshire SHA by the Blair government.

The local Chief Executives, of the PCTs, of the Teaching Hospital, Children's Hospital and Mental Care Trust, and most of the surrounding Trusts, couldn't see the point.  I went  about it in my usual diligent way, talking to people who knew about ISTCs in the Department of Health, in private healthcare providers, barking to a few dogs ("talking to everyone and their dog"), doing the numbers, working it out for myself.  It's not that I'm suspicious, just that I like to check for myself.  The numbers stacked up, I presented my report, and the ISTC was canceled before it left the drawing board.  Of the 38 originally imposed by the Blair government, 2 were canceled, 34 were terminated during the due diligence phase (at a cost to the tax payer of many £millions), and the remaining went into production.

An ISTC worked extremely well in the North West.  It triages patients and prepares them for operations, which are then carried out in NHS hospitals (at least, it did then).  It was brought in with the agreement of all parties, and filled a gap.  It was a success - but one size doesn't fit all! 

Reform of Urgent Care in LondonLondon Ambulance Service

I then did the numbers for a plan to overhaul Urgent Care in the capital - work which subsequently made its way into the Darzi review.  The political strategy this time was right, but it had still been formulated without a proper analysis and could easily have been as embarrassing as the ISTC debacle had been, and the Darzi centres were about to become.  This evidence justified the use of specialist heart centres, on the grounds that patient outcomes were better and ambulances responded to the right cases, even though some hospitals complained bitterly that they wanted the income from patient arrivals.  It made me ever so slightly sick, to think that some hospitals would rather get the fee for an A&E arrival, than let the patient get the best possible care in a specialist centre, care that cost the NHS exactly the same amount.

Lord Darzi in Jan 2009Coping with Darzi Centres

Firmly in the middle of front-line healthcare, I was involved in pulling together GP consortia, one to bid to run a Darzi centre (I think we were too honest about numbers and costs - the (private) company that won it already knew that they would buy in cheap doctors and understaff, on the grounds that no patients ever turned up), and the second to take over the activity of a Darzi centre once its 2 year lease had ended.  The local GPs didn't want anything to do with the centre, and ended up ignoring the building (exactly as the patients had done) but the service was provided cost-effectively in more appropriate and acceptable locations.

AWP, ACP, AQP

The current round of change and opportunities for GPs and the independent healthcare sector (the companies that Mark Britnell is encouraging to buy KPMG consultancy to join in) has gone through a number of titles, and there are subtle but important differences.

AWP - Any Willing Provider is a tacit term for "Anyone already in the NHS or already holding contracts with NHS".  Although anybody is supposed to be able to bid, in practice various reasons have been created to keep the services "in the family".  This upset Blair and subsequently Cameron, so they came out with a new title

ACP - Any Competent Provider was quickly morphed into AQP Any Qualified Provider after the usual round of "how do you know they are competent?"  It was rather inevitable.  However the change is important - basically AQP means that you (the supplier) has to be really determined to bid for a service, with all of the bid costs that entails, because this is about showing on paper that you are competent, not pointing to a track record.  In effect, the GP practices that "just look after our patients" find themselves excluded, and the aggressive for-profit companies can present the most convincing paper cases.  Almost a reversal of the AWP situation, and this is reflected in the legal cases brought by disgruntled losing companies against the commissioners of the services.

So why explain this?

It's important to understand where NHS is now.  The money may be "National", but the providers are in danger of becoming anything but - "International" might be a more appropriate description.

Is this necessarily a bad thing?  Not on the face of it, but small companies each providing their distinct part of the healthcare system can increase the cost of bureaucracy, reduce continuity of care for patients, and start an endless "blame game" when something doesn't work or a target isn't met.  I faced all of this when working with the group of GPs to take over a failing OOH and urgent care service - this is exactly why it was failing.

The NHS does have national standards, but every population is different, every network of healthcare providers is different, and the solutions must, necessarily, be different.  It wold be nice if we could simply take a template for healthcare out of a drawer and say "here is the solution", but just as a GP can't say to every patient with COPD "I don't care what else you have, take this drug to relieve your COPD", so those who make policy actually work have to tailor the solution to the need.

How do you ensure quality in healthcare?

Competition - whatever happened to cooperation?This is perhaps the biggest challenge, when introducing competition in health care.

It is very difficult to measure quality in health care, and even more difficult to measure it contractually and enforce it. One of the biggest challenges is that healthcare is changing so quickly, that we have to learn as we go along. So one month we measuring the level of blood clots, and the next the number of bedsores. This is practically impossible in a contract. Unfortunately, in an environment where money is paid under contract, and providers are driven by the profit motive, there are rather too many opportunities to cut quality and improve profit. It happened under Blair, and it's likely to happen this time around – especially if the comments being made to the US health provider sector, about the UK health market, are anything to go by.

Large integrated organisations, such as UK hospitals, can run extremely efficiently, delivering very expensive and highly detailed and complicated operations, and at the same time providing Accident and Emergency services. With a pool of staff, it's possible to fund the training and experience needed, by performing mundane and routine operations, so that the team is available when something more demanding comes up.

This falls apart when routine operations are taken out of the system and given to an independent provider. That's not to say that a bit of specialisation went improve quality – it often does – but it's incumbent on those planning and making strategy to recognise these challenges.

Keeping track of improving quality

Unfortunately, the so much variety in health care, the maybe very small numbers of each individual type of activity. With small numbers, one complication can completely distort the hospital's ranking, and give a false perception of quality. Because of the small numbers, numbers are aggregated over many years, which means that by the time action is taken against a poorly performing doctor, the figures will take years to recover. An organisation may be unfairly penalised, when they took the action they should have.

Who is capable of enforcing minimum standards?

The Health Secretary has made a big thing of ensuring that the experience gained by the PCTs is retained in the system. In order to commission services, and to enforce the contract, you need a set of skills:

  • understanding the reports that come in from your provider
  • understanding what they mean in context (taken account of small numbers, and the services in the round)
  • the diplomatic skills to raise a query without resorting to the letter of the contract
  • the legal right to demand that the contract being forced
  • the teeth, or financial muscle, to enforce a contract, if this becomes necessary

I don't believe that units based on the size of the GP commissioning groups (effectively, in many cases, the size of the pre-2006 PCT's or smaller) will have any of these. If they're supported by "commissioning support" teams, made up of staff from the old PCT, then they may be able to understand the numbers. But it takes a director or chief executive to have the diplomatic skills; it takes an organisation the size of the new PCT's to have the economic muscle; not only do the hospitals have all of the information, this time round, in many cases, they also employ the community team, so the GPs have nothing to fall back on. For example, if the GPs want to set up a new heart failure pathway, then they will need trained nursing staff; the nursing staff are now employed by the hospital trust, who can easily say "there are no staff available", which prevents the GPs from setting up a new pathway, and means the hospital keeps the old pathway that they favour – and their income.

With teeth?Who is the Regulator, and who is the Commissioner?

Part of the debate has been on the role of Monitor – the foundation trust regulator. Monitors ended up in a ping-pong match – will they be there to encourage competition? In which case, they're not in a position to demand a minimum standards, would withdraw licensed to provide services from someone in an area where there is little competition. Will they be there to encourage cooperation? In which case, what does cooperation mean in an environment driven by profit motive?

Will they be able to enforce minimum standards, then there are also required to encourage AWP (Any Willing Provider), or AQP (Any Qualified Provider)? Given the amount of political interference going on at the moment, will they be able to enforce anything without interference?

The best solution would be local enforcement. Quality, and improving quality, driven by the people for the people.

This doesn't mean elected members on GP consortium boards, because imposing someone on a board that it is designed to make GPs responsible. With resentment and end up being manipulated. This means separating the task of referring from the task of commissioning.

Commissioner should be the local authority, which has a lot of experience of commissioning, and has frameworks in place. Specific health experience can be gained by retaining the best of the commissioners from the PCT's, but as they join the local authority, they will John endowment where the commissioning framework is tried and tested, and well understood.

GPs can get on with doing what they do best; diagnosing, referring patients, and designing new pathways. To be no more conflict between "Commissioner" and "provider", as GPs are responsible for a flowing, and an independent body is responsible for ensuring probity.

And because the local authorities are accountable to the people in their areas, services will be appropriate to local needs, and we'll no longer have to dance to the Department of Health tune.

 

Vince Cable shoots himself in the foot

Clever journalism - logo thanks to Tony DurrantVince CablePoliticians seem to be always walking on thin ice. Some seem to stumble from disaster to disaster, always managing to survive, perhaps because so little is expected of them.  Some fall at the first hurdle, and then are lost forever - perhaps more was expected so they have further to fall.

The campaign by the Daily Telegraph to discredit Liberal Democrat ministers seems to have found a weak point and exploited it.

One after another they tumbled. Much was expected of this – third – party, the last bastion of integrity. Thinking people voted for them at the last election in the hopes that they would bring some of that integrity back into politics.

Then the Liberal Democrats found themselves in power, as the junior partner. Perhaps it was deliberate on behalf of the Conservative strategists, perhaps it was inevitable, but the policies that they were asked to endorse were almost universally the ones that a glib party in opposition would scream for without thinking, but in government they would meet massive opposition.

So the Liberal Democrats are the fall guys, the Conservatives come up smelling of roses.

You could almost feel sorry for them, if they weren’t politicians. It suits the Conservative and Labour party to prolong two party politics, and a strong third party could mean an end to the comfortable gravy train of Westminster, which politicians of both parties have enjoyed so much.  But the Lib Dems know this, and should be wary.

WWJD? What would Jesus do?  Is Christmas an appropriate time to ask this question?

No compromise - forgive a mistake, put right a wrongJesus didn't compromise."forgive your brother seventy times seven" He says at one point, and He was never one to condemn.  But a personal injury, perhaps unintentional, is one thing.  Jesus was not backwards about coming forward about things that were wrong.  He criticised the Pharisees, and He criticised the moneychangers in the temple courtyard.

Integrity is often painful for the person with integrity, but without it, you become "mere bulk and body without breath, cold leg of lamb without mint sauce, cold slops, weak tea - you make me sick" (G K Studdert Kennedy).

The easiest job - an opposition front-bencher

Perhaps it is the easiest job in the world to be the party in opposition, and even easier to be the third party that will never see power; perhaps the Lib Dems got too complacent and have had what was coming to them.

But Britain will be the poorer.  We need our MPs to be a little off balance, with a challenge to serve the people not themselves. Many MPs are of course excellent, but too many got selected as party candidates for favours of the past, and now treat their privileges ias their reward.  There are very few poor MPs, whatever their background.

Perhaps nobody will ever vote for Liberal Democrats again? Let us hope this is not the end of three party politics for Britain.

Winter Olympics - Downhill Skiing is like Management Consultancy

One chance, 100 seconds in 4 yearsYou only have one chance to make a first impression. 
In fact, you only have one chance each time, to make a first impression that sets the scene for that day, that job, that opportunity.
Downhill skiiers get one chance to put in a really good time - this contrasts with some Olympic sports where the competitor re-runs the same activity and the judges take "best of 2" or "best of 3".  This means you can afford to make a mistake (but the competition's still intense - everyone else has the same rules!).  Even in one-on-one competition such as Judo, each competitor gets a second chance to redeem themselves from an earlier mistake.  It becomes a game of "who makes the least mistakes wins".
But in Management Consultancy, you often don't get that lattitude.  One mistake, one slip of concentration, and you are damned. 
Of course you can approach other clients, start other jobs, but can you really afford to lose out?
Athletes train for years to prepare for the Olympics, putting in thousands of hours of intense practice for their 100 seconds of glory.  Athletes train just to attend, knowing that others will almost certainly walk away with the medals.  But it's worth it.
It does take effort, keeping up with the latest developments, focussing our solutions on the needs of our clients rather than on an approach that "should work in theory".  We put the effort in, and I believe the results speak for themselves.

the two-year rule

Self publicist caught outWhy do so many silly ideas become law?
Most of the politicians I know have exceptionally good memories, and studied history. So why do they repeat the mistakes from former years? Perhaps because they've learned the lessons that apply to them, and the mistakes that apply to the general public don't apply to them.
A perfect example of this is seen in both the public sector and the commercial world. A manager or leader has built their reputation on always balancing the budget. They are sent in to troubleshoot failing divisions: they cut spend immediately, usually by terminating all the R&D, and everybody praises them because now costs are lower than income. Then they move on to the next piece of troubleshooting.
And what happens?
within a year or two, the lack of R&D means that this organisation's outputs (services or products) are no longer fit for purpose. It falls to the next manager to take the blame -- the previous self-serving manager retained only the glory of having balance the budget. Of course the previous self-serving manager is also great deal more senior, and is therefore in an even stronger position to make sure the blame lies elsewhere.
You can achieve the same result by terminating overtime, cutting staff, and making everybody work harder. It takes a little while before everybody goes off sick, and people start resigning and finding other jobs. The canny self publicist is well aware of this time-lag, and has moved on quickly. A very clever self publicist will identify where someone else has just done the right thing, and unseat them in time to reap the rewards.[reference available in Freakonomics]
How long does it take for the pigeons to come home?
Typically the effects are visible between 18 months and two years after the change. That's why I call this the two-year rule.
Look around you -- how often have you seen this happen?

"10 High Impact Changes for Service Improvement and Delivery" - NHS Modernisation Agency

AttachmentSize
10HIC.gif4.75 KB

"10 High Impact Changes" presaged the NHS Institute for Innovation and Improvement style of publications - documents written to be accessible and useable for the NHS at large and the public, which gave shortcuts to the achievement of service improvement. It was naturally followed by a whole spate of spin-off documents - "10 High Impact Changes for Primary Care", "10 High Impact Changes in Mental Health", etc
Barbara Hercliffe and I wrote HIC 10 - Workforce, where my main contribution was the case studies. Whole document can be obtained from http://www.ogc.gov.uk/documents/Health_High_Impact_Changes.pdf

Audit: Fitness for Purpose Audit of reconfigured PCT

Determining the commissioning capability of the PCT with the largest in-year deficit in England as it copes post-merger, and with stakeholder and provider changes. The audit both highlighted areas for improvement and triggered some of the required cultural change within the PCT (FFP)

The Measures needed to manage the business

Cascading Targets down to the workers

Not so long ago, the Department of Health was asking “how can we save 40% in management costs in the NHS?”.

I know!  In fact it was a very simple answer.  If the NHS didn’t have to report so many things in so many different ways and in such detail to Department of Health, we’d need 40% less management time (actually, it is pretty close to 40% - very nearly 2 days per week of most line-of-business managers is spent on reporting).

Reporting

 

Of course you need some reporting to run a business.  But let’s make a comparison with a major commercial organisation.

They have a simple rule: Managers only need to know the information that they will use to make a decision.

So the Corporation has strategic objectives (this much income, this much profit, presence growing in these countries).  It delegates these targets to the Vice Presidents, (or Divisional Directors, or whatever you want to call them), and they report back how well they are meeting these targets.  The VP for router sales in Europe reports on his or her total sales, and market penetration, and THAT’s ALL.  Is she hitting target, or not?

Delegating the TARGET, not the TASK

The VP for Europe then turns to her next in line, the managers that report to her.  She delegates a part of her target to each manager.  The country manager for Romania has to increase penetration by 100% (double penetration from 1.1% to 2.2%), whereas the country manager for UK doesn’t get a target for penetration (it’s already at 12% which is about all the market can take), but has to increase profit by 6% (which is a stretch target in this age of austerity).

Service Engineer knows what's expected of him or herEach manager reports on their achievement against target.  And each manager delegates their target to their next in line, breaking it down into manageable, bite-sized chunks.

The service engineer who comes out to see you, the customer, has his tiny component of the main target.  He needs to make 6 customer visits per week and average a customer satisfaction rating of 80%.

There’s an incentive

Everyone has the same incentive.  If they hit target, they keep their job.  If they exceed their target by 5%, they get a bonus.  Although the target is negotiated between each person and their team leader, the team leader has a target to hit so they have to share the whole target amongst their direct reporting staff – so they can’t give people easy-to-reach targets.  Of course if someone misses their target, then they have to wonder how long the company will keep carrying passengers, so there’s a big stick as well as a carrot.  More to the point, because they have a personal target, they (and quite a lot of other people) know what they have to do to hit their target.  It isn’t the threat of losing your job that keeps you hitting target, it is usually clarity – knowing exactly what you have to do, and then getting on and doing it.

Why does it work so well?

One reason it works so well is that everybody has the information they need to do their job and to make the right decisions, but the next manager up doesn’t have too much information – there’s no temptation to micro-manage.  Actually my manager’s manager can see all of my reporting which by rights is for my manager to make decisions, but that’s just so she can check my manager isn’t bluffing.  But in general, there’s a lot less information being reported, so people can simply get on with their job.

We also understand that our targets and achievements add together to contribute to the strategic goals of the organisation.  We aren’t doing pointless things, or pulling in opposite directions to each other, or engaged in political in-fighting.  We’re just getting on with delivering a good outcome.

Where could it go wrong?

My most astute reader will know that I don’t work for a big US corporation – that was some time in the past.  So I have the benefit of hindsight.

This system depends on the top management – they have to know what is possible, what is reasonable, and what is challenging.  They have to know enough not to let country managers or product managers bluff them – argue them to a target which is too easy.  They have to make sure that everyone can see that the system is fair. 

What could possibly go wrong?  Penguin with cymbals on a sleeping polar bearBut if we look at our beloved public services, the senior management are very experienced and generally realistic, and set demanding targets with their eyes wide open as to what investment it will take.

Why doesn’t it seem to work in UK public services?

Now there’s a question and a half.  Again, I’ve worked in a number of organisations and consulted into even more. 

One or two reasons for this stand out to me.

  1. The US corporation demanded business cases for the year ahead, prepared in July.  These were then examined, and if approved, we heard back by September.  We then had 6 months to come up with a workable plan and investment structure to prepare to deliver our proposal successfully.  Contrast that with NHS over the last 10 years – we don’t know the financial currency, let alone the budgets, until 3 months AFTER the business year has begun so we scrabble to plan well after the work should have started
  2. There’s no carrot, and no stick.  The wrong people get praise, and the wrong people get punishment.  There’s no room for bonuses for anybody except the directors, and they often negotiate targets that mean they get their bonus whatever happens to the organisation.  And job losses happen not because of performance, but because of changes outside of people’s control

Can anything put this right?

Well, GP commissioning might be a good thing. 

GPs are independent contractors – if they get their sums wrong, then they stand to lose a lot of money.  Typically the partner share in a GP practice is around 1/3 of the total income for that practice (and still only adds up to the equivalent of a good GP salary for each partner).  So a GP practice can be thoroughly inefficient, right up to spending half as much again as the average practice on surplus staffing, empty clinics and major inefficiencies, and the only people who suffer are the GP partners.  If they are the cause of the inefficiency, then it’s entirely right that they suffer.

But good GP partners employ a good practice manager, who in turn puts in place good team leaders to make sure everything runs smoothly and efficiently.  And they are going to demand the same from their suppliers, whether these suppliers are great big hospitals or small charities providing counselling services.

The other major incentive for GPs is that they are here to stay.  They own a business that is entirely local – the GP income is dependent on the number of local patients so if they upset the local customer base, they can’t relocate and start again (easily).  And the GPs are here for decades; they aren’t on a salary and taking the next career move.  So decisions they make about patient care are important – mistakes will come back to haunt them in 10 years or 15 years.  So they have a pretty big incentive to get things right.

All we need now is for GPs to buy in the contracting and contract management skills to manage suppliers much bigger and more experienced than themselves, and there’s a pool of PCT contracting and contract management skills but sadly in some cases they haven’t exactly proven that they are worth buying in.

Minney.org

Not for nothing Minney.org has a trading name “the Social Return Company”. 

We specialise in delivering the very best for patients, for people receiving care, for people receiving support, for people at the vulnerable points in their lives.  We help with understanding the needs (in the round, not just “which pathway should we improve now?” but “which change lays the best foundation for the next change? and how fast can we move without too much risk?”).  We specialise in setting the right CQUIN or other incentives and enforcing them, and in engaging the negotiating parties to agree these incentives.  We put in place contract management so you don’t need us any more, contract management using your own staff and expanding their skills.

We’re always moving forwards, so we enjoy transferring the knowledge so that people can do the job, and we enjoy learning and delivering and teaching the next challenge.  Right now, the challenge is building a new NHS, and a new Social Care.  And that should keep us all busy for the next few years.

 

Measuring things - using statistics and analysing data

This is currently a holding page to take you to sections on Analysing your results
and Basic descriptive statistics including an explanation of what data are and some of the most common tests used to analyse data.

Descriptive statistics and basic statistical tests

The key function of this page is to outline the main statistical tests which you'll use for comparing data.

Descriptive statistics and basic statistical tests

In these two sections, to aid understanding

Descriptive statistics

First thing to do with the data is to plot the dots

 

It’s worth explaining some simple things about data:

 

Terminology: Analysis is all about testing hypotheses (what you think might be happening). The null hypothesis (H0) is that any variation is due to chance. You can have any number of Hypotheses (H1 . . Hn) of specific things happening, though usually we just look for situations where deviation from the null hypothesis is unlikely to be due to chance.

What do I mean by that?

If you know that the national average for a baby’s weight at 6 weeks is xx kgs, and 20 babies recorded by this one midwife during this year give an average 0.2kg less, then does this mean there is a problem or could this be due to chance?

Depending on your level of risk, you might say “if there’s a one in 10 chance that it’s not due to chance but something is going on (the mothers are all not feeding their babies properly, the midwife isn’t giving the right advice) then I want to investigate”, then you would look at the probability that this is due to chance and act accordingly.

Most statisticians use 1 in 20 (5%, <0.05) and 1 in 100 (1%, <0.01) or even 1 in 1000 (0.1% or <0.001) chance. I believe there are a great many situations in ealth where 1 in 10 (10%, 0.1) is a reasonable point at which to take action.

Types of Data

Numbers

Real numbers (numbers of people, numbers of minutes, heights, numbers of activities) are what are called parametric. That is they are real numbers. This is crucial for using statistics – many are designed for using only with parametric data.

In the natural world, numbers tend to follow a normal distribution, as someone described it “a small amount at one end, a lot in the middle, and a small amount at the other end”; or in the case of height, most adults are roughly around average height, with only very few substantially shorter or taller than the average.

To look at whether the numbers in your population are normal (also often a requirement for statistics), plot the data as columns in categories or groups of data. It should form a bell-shape.

Histogram of Height in a population

 

Skewness

Times tend to plot as a skewed curve. Hardly anyone gets to see the clinician in no time at all (I’ve come across this occasionally, where the way the data are recorded, it’s the clinician who makes the decision whether a person should see the clinician, and of course sees them immediately. Watch for this as it’s not evidence of short waiting times, only of poor recording). So hardly anyone sees the clinician in no time at all, most people see them within a short period (whether that’s minutes or days), and perhaps half of the population don’t see the clinician quickly and sort-of tail off into the distance.

Numerically, the analyses to use are Kurtosis and Skewness.

Start with the Mean (average – MS Excel function =Average()) and Standard Deviation (amount of variation from the Mean – MS Excel function =Stdev()).

Categories

Categories are not numbers. The categories are mutually exclusive, such as Male or Female, Success or Failure, Sheffield or Barnsley or Doncaster.

You might have data for attendance at the Urgent Care Centre (UCC), and capture where people have come from. You want to know if referrals on to A&E are what could be expected.In other words, if three times as many people come from Sheffield as from either Barnsley or Doncaster, are the same proportion of people being referred to A&E or do they have different habits?

Use Chi squared (χ2) for this. This works out what you would expect from the totals, and compares this with what you actually have

Categories such as heights, as shown on the above normal curve, should not be used in this way as the categories are connected in a numeric way – they are on the same scale.

For example if you worked out how far people travelled to get to the UCC that would be a numeric score.

Combinations of categories and numeric data allow you to run useful tests such as Student’s t and ANOVA.

Ranks

Where you have information which isn’t numeric but which can be ranked, for example satisfaction, then you should not treat it as a numeric score but as a non-parametric ranking.

You can also convert numeric scores to ranks, eg “closest, next closest” or “first ten, next ten” which sort-of “normalises” skewed data.

This is an excellent way to convert essentially qualitative data into quantitative.

Although the data are non-parametric not parametric, you can still perform a number of numeric statistical tests such as correlations.

Percentages

Note percentages are NOT parametric or numeric scores.

Because they are proportions not absolute numbers, they don’t always fit the proper definitions for populations used in statistics, so be careful interpreting them.

Run the usual tests but make a note that there could be a problem.

 

Basic statistical tests for analysing data

The key function of this page is to outline the main statistical tests which you'll use for comparing data.

They are:

Parametric tests

Where you are using actual numbers, the numbers of people coming, number of minutes wait, etc and you can reasonably expect a 'normal' distribution (you've no evidence that there is some bias), then use Parametric tests

Student's t-test

compares two averages and determines if there's a real difference between them or if the difference is just due to natural variation (no statistics give absolute answers, they just give a level of confidence. One chance in 10 (10%), one chance in 20 (5%) and one chance in 100 (1%) are usually taken as the border lines for confidence that the difference, or similarity, is reliable.

Student's t-test was designed by William Gossett to cope with smaller numbers (usually taken to mean samples of fewer than 30 points) because the z-score starts to get inaccurate with smaller numbers. The t-test works for larger numbers too

Anovar

Technically the F-test, this compares any number of averages essentially by comparing them pairwise and reporting if any are statistically different. Same as the t-test, this assumes that the distributions of the data are similar

Anovar is short for "Analysis of Variance" and the F-test is named after R.A. Fisher. Anovar is usually applied to one-way data but applies equally to more ways (see below for note)

Correlation

requires that each measure has at least two bits of data attached; for example each week of data could have: number of users; average wait; number of staff hours; average daily temperature. You're looking for any association between any of these bits of data for example do waiting times go up when numbers attending go up, or down when staff hours go up. The easiest test to use is r2 (r-squared) or Product Moment Correlation Coefficient (Pearsons).

R2 values are between 0 and 1. The higher the value, the stronger the correlation. You'll always get a straight line between two points (i.e. for two points r2 will always be 1.0) so be sensible - with numbers of points below 30 you should be careful interpreting the results

Derek Rowntree lists how you might describe correlations for r, so I've updated it here for r2 as follows:

R2 value

 

Degree of correlation or association

0-0.16

Negligible, weak or low

0.17-0.5

Moderate

0.51-0.8

Strong

0.81-1.0


Very strong

Note correlation doesn't indicate causation. There's a correlation between a person's weight and height, but extra weight does not CAUSE extra height. It's probably that extra height causes extra weight but you can't tell that from finding there's a correlation between the two.

Non-Parametric tests

The first tests listed are so-called Parametric tests, that is they work on absolute values - how high, how long, how heavy.

Non-parametric tests work on scores, such as % (the proportion, how many <b>of</b> how many, limited to a range usually 0 - 100), the rank (first, second, third - with no indication "by how much") and so on

Ranking can be useful if you can't assign a value, e.g. if people put their preferences in order.

Non-parametric tests should also be used if you've created artificial scores, such as ranking patient satisfaction

Chi-Square test

applied to two-way or more data (and one of the most popular tests in social sciences) - for one-way samples either use absolute values or compare Standard Errors of Proportions. Chi-Squared compares the observed frequencies (proportion * total) with the expected frequencies (average proportion * total, calculated from both totals row and column)

Spearman's Rank Coefficient

Essentially the same as the coefficient of correlation (and shown as a p2 value) but using ranks instead. Interpret the same way.

All of the above will be covered in more detail on accompanying pages

Footnote 1 - one-way,two-way and more. One way for a comparison of averages means that you're only comparing one difference between the groups, or populations. Men vs women. Trees vs Bushes vs Grasses. Red vs Blue vs Green vs Yellow vs Black.

Two way means you are comparing in two different groups, for example


 

Men

Women


Children

 

 


Working age

 

 


Elderly

 

 

and more ways just makes this more complicated

It's worth getting a simple book on statistics e.g. Derek Rowntree's "Statistics without tears"which will give you a general background - this would take too long on a web page when there are so many excellent resources out there. Other books on statistics can be found on this Statistics Resources

Easy Statistical Tests for Analysing Data

The most common ways to analyse data you've gathered from measuring results

So you've got your numbers, how many people, how much was done, using what resource; now you have to report it in a meaningful manner.
This is where Statistics fits in
If you compare two numbers, e.g. the number of users before the new service was introduced and the number after, then you can get one of three results:
MoreThe second number is higher than the first
The SameNo Change
Less
This doesn't tell you whether you could expect the same change if you took measures a second time; for example you might have more people one week completely at random.
To counter this, you can use statistics, which estimate the 'likelihood' that one number is different from another, or the likelihood (H0 or Null Hypothesis) that they are the same.
Of course you need to go back to What constitutes success, or the benefits outlined in the Benefits Approach. To make the reports meaningful, you need to decide what they will demonstrate - 40 graphs and charts pinned to a notice board are pretty meaningless.
For example, if you want to show the service has improved and you've collected:
numbers of people using the service each day
average wait from arrival to being seen
average wait from appointment time to being seen
period from referral to appointment
patient satisfaction
follow-up or repeat appointments as a proportion of total day's appointments
number of that day's appointments that result in a follow- up
'see and treat' resolution of problem that day
which of these would show an improved service?
None! Comparing the figures with the week before, the month before, the year before would show if the service was improving or not (though it's important to remember to use run charts or averages - figures comparing one day to another are counter-productive).
Of course you also need enough clinical knowledge to know what direction of change represents an improvement - follow- up appointments may be good, and they may represent failure.
How do you analyse the data
Plot the dots
The first thing to do is to understand your data. What does it look like?
Is it "normal" or not?
parametric or non-parametric?
Does the data tell you what you need to know, or do you need to process it in some way before it will?
We'll explore Basic descriptive statistics and simple statistical tests in the next page

BOOK: Ward & Daniel "Benefits Management: Delivering Value from IS & IT Investments"

The problem with the IS/IT investment is that people often don’t think about the benefits - "of course we need it". Ward and Daniel illustrate this from a survey reported in 2003 (Lambert and Edwards – also at Cranfield University) as follows:

·        55% of respondents think that an appraisal of the IS investment is important

·        Only 22% have a process to perform this appraisal

·        Only 10% consider the implications of business change from their IS investment

·        And ultimately – only 27% of projects appear to justify the investment made

This is surprising, given how important investment appraisal is in the preparation of the business case. It begs the question: are honest business cases prepared for IS investment?
 
The business case – "benefits cost analysis"
The business case is designed to explain the financial justification for any investment of resources. Examples of approaches include:
·        ROI - Return On Investment
·        NPV – Net Present Value
·        ROA/ROCE – Return On Assets/Return On Capital Employed
·        And many others, that may illustrate a non-financial return to a resource investment
In principle of course, the business case will illustrate what the reward is compared to the risk, and what the return is for the resources engaged. Typically a business case will make a case to: do things better, do new things, and/or stop doing things.
 
Working with stakeholders
Very little gets done unless you involve people. Ward and Daniel into some detail on how to engage stakeholders; in particular took about a stakeholder perspective, planning the business change, and valuing the benefits.
One point make repeatedly is that the value from IT investment comes from the business changes – "IT enabled change" not just the technology. Therefore it is critical to involve the business managers, and to identify, and satisfy, their business needs. Technology can help to integrate processes and coordinate resources (Value Linking), to increase the speed at which the business performs (Value Acceleration), and to realign job roles and organisational structures with valued processes (Value Restructuring).
Therefore it's important to identify the stakeholders for the project (who's relevant).
Stakeholders can be categorised in a number of different ways, including their influence, apart make decisions, and their attitude towards the project, and many of these methods are illustrated. Having identified the stakeholders, identify what the benefits/dis-benefits ("what's in it for me?") are for the most important, and what change activities will be necessary, and from this identify where the resistance may come.
It isn't always the explicit changes that create resistance. Often it is a secondary or perceived change such as:
·        Increased accountability, less discretion and autonomy
·        Individual performances more visible
·        Having to rely on others to achieve performance
·        Collective rather than individual recognition/reward
·        A learning curve
Stakeholders can be approached in a number of different ways, with varying results:
Ward & Daniel - Management Approach and Organisation Mode
In most organisations, substantial investment and change decisions are made by senior management. Typically decisions are justified with a rational presentation (eg a Business Case), which clearly fits into the top left-hand box of the diagram above.
·        The solid arrow in the illustration above shows management approach following the organisational behaviour. The decision to proceed with the investment is made at a senior level. The project then works with (‘coalition’) the business to build trust. It is implemented by finding quick wins and self-interest, and minimising dis-benefits to bring about the business change.
·        The blue dotted (top) arrow follows a rational approach. By not building trust that the top management has the interests of the whole business at heart, divisions and teams will seek to avoid blame, by removing business change that represents a risk. Inevitably this means that the final solution involves less change and therefore less benefit (note that not all change is a benefit, although all benefits involve change)
·        The yellow lower arrow illustrates solution imposed without discussion. Once this imposed solution reaches the teams who need to make the changes, each team will create their own objections, and the delays will result in individual negotiations resulting uncoordinated and inconsistent solutions, which will miss many of the overall benefits.
 
Implementing a benefits-led approach
Most decisions are already made on the basis of “benefits”, but often the only person who understands this is the person making the decision, and it isn’t written down.
Why would you change the way your business works to use a Benefits-Led approach? It can be for either tactical reasons (precipitated by a crisis; recognising business is disenfranchised from decisions over investment; a way to reengage business), or strategic (decide to improve the process or to improve governance standards).
What does it actually mean? Decisions about investment in technology inevitably, and always, affect people – as Ward and Daniel put it: "techno-social". This means that they aren't simply a decision about money, they are a decision about changing the way of working. Changes to technology will affect the business, in one way or another (after all, otherwise why would you bother investing?), the business, and the people in the business, need to play a part in making the decision.
How? Implementing a benefits approach will depend on the context. Ward and Daniel give three example contexts where thought needs to be given before implementing a benefits led approach:
·        Public sector – multitude of stakeholders (the agenda is not just financial viability, but also service to the population, and an ever-changing political agenda); and frequently a mandatory requirement that has to be met (but it won't be met by mandating compliance, it still requires a consensus to make the relevant business changes)
·        Small business – small businesses don’t typically have specialist project managers and benefits managers. Everyone is involved, and everyone contributes. The benefits led approach needs to recognise the specific benefits for each individual, as tangible rather than nebulous (for the greater good) benefits
·        Large organisation, multi-site – although the sites may appear to be clones of each other, the benefits identified for one site may not apply to every site. For example, each site starts from a different point, and they have different stakeholders. It is tempting to do the benefits planning once, and then roll it out on this basis, but some form of benefits planning needs to be done with each site in order to build consensus and agreement for the changes as they apply to that site, otherwise we will end up with the illustration of the yellow arrow – different sites create different objections requiring different work-arounds, which means systems that don’t work in harmony
 
Summary
The book is a good review of a vast amount of academic literature to date (2006), and used in Cranfield University to teach the most widely taught benefits approach in the UK.
To my mind, it may miss the point in some areas, especially the use of the highly regulated and formalised Benefits Dependency Network which can be very difficult for line with "common-sense". Bradley's version of the benefits dependency network takes a very different, and far less formal approach, and I prefer it.
 
 

Minitab 16 – statistics for infrequent users

It isn’t often that I write a piece which blatantly advertises a product. But a friend had to get his Masters dissertation in on time, and was fighting with the University-issued stats package, so I turned to my favourite Statistical Analysis software.
It’s my favourite because:

It does all of the statistical functions I need, in a clear and sensible way (yes you can do them in MS Excel but you have to jump through hoops)
It does Run Charts and Quality Management out of the box (you have to buy additional modules for SAS, SPSS, etc)
It’s low cost/ low maintenance (It is designed for working people rather than academics – I’m a working person with academic pretensions)
It has a really good help system

I’ve been using Minitab (www.minitab.com) for years, but I was still stunned to see this new feature – Assistant.Minitab Assistant Drop-down menu
 
 
 
 
Plot the Dots
(to quote Davis Balastracci)
When I click the Graphical Analysis button, it opens up a flow chart. This looks like a useful help screen, but it turns out that it’s live – it looks at your data for you! Of course you can plot graphs in Excel, but do you know which graphs to plot? 
Test your hypothesis
This is where the rubber meets the road. What am I trying to test, and am I right? I mean, look at the choices:

Compare one sample with a target
Compare two samples with each other
Compare more than two samples
Testing a hypothesis - the decision-tree in Minitab

ANOVA (Analysis of Variance) for continuous variables, and Chi-Square for discreet groups (male/female, ethnic group, etc) are my favourites.
It does reveal its working roots – I found that I had to use Chi-Square % Defective to test whether the balance of experienced/new staff differed between the company subdivisions (so if you are experienced, you are defective – actually that could be right?).
Looking at the results
But the way it presents the results had me excited! As an infrequent user, I get to stare at the Probability, F ratios, T ratios and everything else and think “is bigger better, or is it smaller that is better?”
Here’s what look like a set of dots where the human eye would say “OK I can see the trend, but it isn’t very close”Presenting the results of a regression line in Minitab
Minitab tells you (from the Assistant) not just the Probability that there is a relationship (nice bar chart with “degrees of blueness”), but also that in spite of the high probability of a relationship it only represents 26% of the variation (degrees of greenness) and shows the correlation too. All on a single sheet.
Go and download your own trial copy: www.mintab.com (yes, Quality Companion is about service improvement and lean – that’s worth downloading too) 

Plot the Dots

The first thing you should do whenever you find yourself with data, is to visualise the data - typically to plot it on a graph.
Which graph you use will depend on what data you have.
For example, suppose you have
(NOTE EACH OF THESE SECTIONS IS EXPLAINED IN MORE DETAIL ON THE LINKS)

Scores, or Numbers against departments
For example, this could include:

numbers of service user contacts per person (NOTE if you have numbers of contacts per week for a number of weeks, see below)
patient satisfaction scores for each department or organisation
smoking quitters per 1,000 population by GP practice
average length of stay (LOS) by speciality and by ward
income (in HRG) per consultant

Plot
a histograph or histogram
If it's a normal curve, you can boxplot the averages and if there's something
to go on then compare averages with
Student's t test for two averages
ANOVA analysis of variables for more than two averages

For a two-way analysis (where you have two comparators eg sex, hospital or speciality, ward)

use ANOVA analysis of variables for two or more ways
use Chi-squared () to test whether the observations are significantly different from what you would 'expect'

 

Change over time
For example:

waiting time before a major change in procedure such as service redesign, and after, with scores collected weekly over a 10 month period
max heart rate during exercise (per session) for a patient going through pulmonary rehab over a 14 week period
travel time to work before, during and after a road widening scheme

Draw a Run Chart
If it shows a change, then you can consider comparing averages using typically Student's t test for the average before and the average after

 

Compare pairs of values
For example:

BMIs (Body Mass Indexs) vs Blood Pressures of a whole lot of people
Cost vs Tariff price
Patient Survey satisfaction score vs Practice List size for GP practices

Plot a scatter graph
if you can see a pattern emerging in the scatter graph, then

calculate the Correlation Coefficient and regression line
possibly plan to do root cause analysis or design an experiment to determine if there's a causative relationship between the values

Plot a bubble graph or Boston Matrix
Only use these fancy illustrations if you have particular kinds of data,
for example Opportunity/ Cost for a series of projects. Bubble graphs can be used to show a third dimension of data (eg adding the waist circumference to the BMI vs Blood Pressure graph) but generally makes it all look very complicated with more than about 10 data points.

 

Plot the dots - histogram normal curve

Lets assume you have a column of names, with numbers next to them, eg:
Scores, or Numbers against departments
For example, this could include:

numbers of service user contacts per person (NOTE if you have numbers of contacts per week for a number of weeks, you may get a better view of the data by looking at a <run chart>
patient satisfaction scores for each department or organisation
smoking quitters per 1,000 population by GP practice
average length of stay (LOS) by speciality and by ward

Plot the dots
First step, as always, is to Plot the Dots. In this case, plot a histograph or column graph of the data grouped into columns.
You'll need to group your data into evenly spaced categories, and then count the number in each category. May I suggest no more categories than half the number of items, and no more categories than 15 if there are a lot of items?
Grouping into categories
In order to work out how wide the categories should be, work out the range (difference between maximum score and minimum score), divide by the number of categories (eg 10), and this is the width of categories.
It's relatively easy to do this in MiniTab, which is my preferred route (copy and paste the data from Excel, then strong>Graph
menu/ Histogram: With Fit [OK]. Then select the variable (numbers column). In Labels... give it a title and add a footnote. In Data View... select a distribution curve.
I suggest Normal or Lognormal.
Then from the histogram produced, right click on the bars and select Edit bars. The section you want is Binning which is the term for allocating numbers to categories. You'll see what you can edit from this point. It's automatically produced what we want already.
Instructions in:

Excel 97-2003
Excel 2007
Minitab 9
SPSS 16

Histogram_of_PCT_Popn_2007.gif
You can straight away see that the graph above doesn't fit the data very well, and it actually looks like a lognormal (lots around the 250,000 mark, but some right up past the 1,000,000 mark), which when I replot using the lognormal distribution curve looks like this:
Histogram_of_Popn_2007_lognormal.gif(incidentally for those who spotted the Loc and Scale above, e12.59=293,000 which is a closer "average" (mode) population than the one predicted from the raw numbers). In the real world, heights come out as normal distributions (because both very small and very large are relatively rare), but a great many numbers come out as lognormal distributions (because:

you can't have populations, or times, of less than 0 but you can have them extending to infinity
mostly we set a minimum but a maximum happens by accident)

What does this tell you?
That most scores are somewhere in the middle, which is what you'd expect.
Skewed data
If you get a big skew (like the one above) then you may need to transform your data to take account of bias in the method of scoring (eg a logarithmic transform or a rank).
Two humps
You have two real populations

population heights, which are likely to consist of a hump for male heights and a hump for female heights). Histogram_of_Height_2.gif
Length of stay, two populations those without complications and those with complications
etc

Note you need quite a large number of data points to tell whether you actually have two humps or just some random variation
A square
If your column graph comes out like this:
Histogram_of_Age.gif then there's probably not a lot you can do with it, as it doesn't form a normal distribution

Process Redesign using Care Pathway Simulator

Simon Dodds, a surgeon in Good Hope Hospital, developed an excellent piece of software to model the flow of patients/service users through a care pathway given constraints of staffing, equipment and facilities. It can be used to determine whether investment in additional resources is warranted and what will happen if you don't - it was sparked when Simon showed the consultants at the hospital that the reduction in length of stay (LOS) they needed was not "lots", but 0.5 day for a LOS of 5 days (much more achievable) to reduce elective surgery cancellations from 70% to less than 30% (cancellations due to no bed being available because of emergencies).
Simon turned this into a book (Three Wins: Service Improvement using value stream design) and a software programme called Care Pathway Simulator which I used for cross-referencing my own computer modelling for redesigning both ECP use across England and urgent care supply in London. It gets a mention here because I get an acknowledgement in the book for the contribution I made in the early stages.
Note Simon Dodds also offers a course in Value stream design and computer simulation (see the www.3-wins.com website)