Answer:
When they are caught, they are thrown away. When they escape, you itch all day.
My first half is normally made of stone, my second is crazy, and I taste sweet when complete.
The itsy bitsy teeny weeny yellow polka dot variety is quite popular.
It is weightless, you can see it. If you put it in a barrel it will make it lighter?
I'm the source of all emotion, but I'm caged in a white prison.
I appear in the morning. But am always there. You can never see me. Though I am everywhere. By night I am gone, though I sometimes never was. Nothing can defeat me. But I am easily gone.