So the verdict is in, the evidence conclusive; I am wrong. The vote was 8 to 6 in favor of using the phrase it's all downhill from here to mean that things will get easier. It doesn't mean that I'm going to use the phrase correctly... I'll probably start using it with both meanings, leading to lots of confusion for everyone. Yea for that, right?