How do we know the vaccines are actually working?

I have been gamely trying to understand just how we know that the COVID-19 vaccines are actually working. I read reports and articles assuring me that that is the case, but how do we actually know?

Surprisingly, it's not easy to ascertain. Perhaps the best overall article I have found about how efficacious and effective the new vaccines are is this one, produced for the UK Parliament. The first thing to note, then, is the difference between vaccine efficacy and effectiveness.

Vaccine efficacy is what is measured during randomized controlled clinical tests, before a vaccine is licensed and released, which compare the rates of infection between vaccinated people and those receiving a placebo. This I understand, and this is where figures like 94% for the Pfizer/BioNTech vaccine come from. It is a kind of theoretical measure, based as it is on mainly healthy younger volunteers and only over a relatively short timespan.

Vaccine effectiveness, on the other hand, is a more real-world measure, evaluated during post-license observational studies of the whole population of vaccinated people of all different ages, health profiles, etc, over an extended period. This may be quite different from the theoretical efficacy rate, and this really is the one that counts. But it is notoriously hard to pin down. How do you measure it in practice? Is it just the proportion of vaccinated people who catch the virus compared to the proportion of those unvaccinated people who test positive?

The only published study I have seen so far comes from Israel, and concludes that the Pfizer/BioNTech vaccine is around 50% effective in preventing transmission, and just 33% effective for adults over 65 years of age. Scientists seem pleased with this, but it is light years away from the 94% theoretical efficacy. Britain, which also started vaccinating quite early, says it will not be able to produce any useful effectiveness figures until late February at the earliest, although an early analysis suggests that just the first dose of the Pfizer vaccine reduces the risk of infection by around 70% (57% on those over 80). A Scottish study has shown an 81% reduction is hospitalizations among vaccinated seniors over 80. These are, however, not very exhaustive or conclusive results. 

So, what are the (mainly non-scientific, political) claims that "the vaccines are working" based on? Just the fact that new cases have been falling in recent weeks? (I had hoped this was due to public health measures and the lockdowns we are all putting up with.) The recent fall in hospitalizations and deaths? (Is that not because we are getting better at treating the infections that do occur?) 

What is the so-called "real world" data that many of the more cavalier scientists are touting? Merely comparing the hospitalization rates of those who have had the vaccine with a completely different group of unvaccinated people, who may have different health, age and socioeconomic profiles. This is hardly controlled, double-blind testing.

Then, of course, there is the whole issue of whether the vaccines are "sterilizing vaccines" (ones that actually prevent infections and transmission completely) or "disease modifying vaccines" (that just reduce the severity of the disease when it is contracted). There is the issue of the duration of the immune response. There is the whole issue of what we are looking for in a vaccine: a cessation of transmission and carriage of the virus, or just a reduction in the numbers of hospitalizations, or of deaths? The whole field is rife with gradations and definitions and assumptions and what have you.

Everyone claims to be "following the science" and "basing decisions on the data", and yet we have widely different policies arising, e.g. some countries and provinces are happy to give AstraZeneca to seniors, while some are not; some are waiting the manufacturer-prescribed three or four weeks (or up to six weeks) between doses, while some are waiting three months, even four months, in between. All still claim to be "following the science". At this point, "the science" becomes a very murky and imprecise concept indeed.

The bottom line, though, is that any functional vaccines are better than none, and I'm certainly not going to turn any down. Some people will, though, and there have already been differences in what some countries are willing to accept in a vaccine (e.g. South Africa has rejected the AstraZeneca vaccine because it has not been shown to be effective enough against the South African variant, and France and Germany are not using it on over 65s because of a lack of testing on that age group, despite some evidence from the UK that it is effective on seniors). It's a real scientific minefield.