How do we decide which statues should be taken down?

As I have noted before, we are going through a bit of a reflex moment in which statues and monuments are being torn down wholesale, and buildings, roads and whole organizations renamed, largely in response to the demands of Black Lives Matter and similar anti-racist organizations. And, as I have also noted, not everyone is happy (is everyone EVER happy?), not even all black people.
Some statues are being taken down, but not necessarily for the right reasons. Take, for example the "temporary" removal of the statue of Christopher Coumbus in Chicago, which was ordered removed because of violent protests (the violence came from both protesters and police), and not for any high-flown ideas about educating the public or furthering the cause of anti-racism. Protesters are claiming it as a victory, gushing that they are "proud to see the removal of a statue that represents white supremacy, as a win towards the goal of decolonization". But, whatever you think about their conclusions, is the threat of violence really a good justification for anything?
I understand the theory behind it all - at its simplest, that we should not be celebrating individuals from history that may have had racist views or even participated in the slave trade back in the day. Set against that are people who argue that the past was a different country with different attitudes and norms, and people who had otherwise laudable achievements should not be judged according to modern standards that were not applicable then.
Well, make of that what you will; another consideration, though, is just how practical an action it is, not to mention who should make the decisions, and how far it should go. I have already discussed elsewhere the case for removing a statue of Gandhi, but as John Ibbitson describes in an article in today's Globe, almost no historical figure is wholly exempt from critisism of some sort, however much good they may have done in other respects: Margaret Sanger was a champion for women's reproductive rights, but she also embraced eugenics; ditto women's rights pioneer Nellie McClung; Egerton Ryerson, pioneer of public education in Canada, was complicit in the residential school system; influential prime minister William Lyon Mackenzie King chose to lock up Japanese Canadians during the Second World War; even revered NDP leader and father of Canadian medicare Tommy Douglas spoke out about gay people being mentally ill; etc, etc.
Do we rip down ALL of their statues, rename all of their roads, buildings and community centres? If not all, then which? Who actually passes the test of sainthood (hell, even Mother Teresa has been accused of forced conversions and links to colonialism and racism)? And who gets to decide? And will it actually do any good anyway?