"Why did my rank in phantom castle decrease? From 9000, to 13000? Was it because of the number of defeats I keep getting, or I'm not playing that frequently?"

You should play it frequently as other players go up in higher ranks. Try to defeat stronger enemys and increase your ranks.

Defeats in themselves do not lower your rank as you still gain points for them so potentially your rank can still go up. With defeats it does lower the amounts of points you will get on your next win so be aware of that. It is more likely other players have been playing a lot and have gained more points than you thus lowering your rank.