I may not be able to answer these questions satisfactorily for the non-Christian, because understanding God means having faith even while knowing that you will never understand God fully. I am however going to attempt to draw on some scripture and its insight in order to answer some of these questions in my own mind, and in so doing, hopefully help some others out with their questions.
The first scripture that comes to my mind, always, in times of turmoil is John 16:33 in which Jesus says "I have told you these things, so that in me you may have peace. In this world you will have trouble. But take heart! I have overcome the world." This lets me know that things will not always be easy in this life; trouble will inevitably come to me. It does give me peace to know that my Savior has overcome this world and that by trusting in him, I will be victorious along side him one day.
This doesn't always help me with immediate problems, though. Sometimes I need help just making it through a struggle. Jesus says in Matthew 5:4 that those who mourn will be comforted. There are times when my prayers aren't answered as I wanted and I just need to be comforted. How would I know how to comfort other, as I am called to do in Romans 12:15 if I have never know what it is like to mourn? Jesus felt feelings of loss and sadness too, as he did when Lazarus died and he wept for his friend. If God allowed Jesus to experience this pain, it must have been for a reason.
I know one thing for sure and that is that God does not punish his children because they didn't pray hard enough. Or yet because they did something to deserve cancer, or homelessness, or poverty, or depression, or any number of things that cause us pain in our lives. If that were the case, then we would all be living in poverty and dying of cancer because there is no one deserves any blessings more than another. We are told in Romans 3:21-26 that everyone has missed the mark (sinned). We all deserve the bad things, not the good.
So, why pray at all if God is just going to allow whatever is going to happen, happen? I pray because I am called to. In Romans 12:11-13 I am told to be joyful in hope, patient in affliction, faithful in prayer. This is one way I demonstrate my obedience and faith to God, knowing that I am undeserving of his grace, yet hopeful that he will have mercy on me.
Alas, some things that I pray to be relieved of will not be removed from my life this side of heaven. This is where the faith comes in. I have to trust that God sees around the corners that I am blind to since his ways are higher than my ways (Isaiah 55:7-9).
I can't say I wasn't expecting trouble, because Jesus did warn of it. I must accept it as a lesson in humility and benevolence because I want to be able to comfort those who need our comfort, just as Jesus taught us. I consider it pure joy.
2Consider it pure joy, my brothers, whenever you face trials of many kinds, 3because you know that the testing of your faith develops perseverance. 4Perseverance must finish its work so that you may be mature and complete, not lacking anything. 5If any of you lacks wisdom, he should ask God, who gives generously to all without finding fault, and it will be given to him.