How soon after a cat gives birth do they leave the nest?

i'm worried my cat doesn't have a normal bond with her kittens. She gave birth to her kittens 5 days ago. She has 4. The day she gave birth, she left the nesting box. Last night she slept in my room instead of in the nesting box (in my 20 month olds room he is still in a crib) with her kittens. She leaves them every opportunity she can. We have 2 other territorial female cats and are concerned that her neglegence make result in their demise. Could she be suffering from postpartum kitty depression?