I never thought about it in this manner. I "own" my brain. However, the answer to your question is yes, in part. Our brains do not decay so-to-speak. As we age, the things we were able to remember, we forget, our reflexes aren't as quick, etc. It is our brain functions which decay if this is the word you choose to use, not the brain itself.
Decay is the act of decomposition by bacteria or fungi. Technically, decay is caused by "detrivores" therefore the matter consumed would have to have died. Therefore, no. However, bacterial infections and viral (including prions and virion) could cause destruction and and death of neurons.