Hi alkay,
This is expected default behavior. When you disconnect from a room, RealtimeAvatarManager will no longer be connected to the server, so the avatar is destroyed.
Normcore allows you to use multiple instances of Realtime so you can be connected to more than one room at the same time. Take a look at this post with some pointers on how to set up multiple rooms with additive scene loading, and how you can persist your avatar between those scenes.
Nick