The reason - Cats could pass a harmful parasite named toxoplasmosis in their feces. The dangerous parasite could cause severe brain damage to the unborn baby. There is good news - handling and feeding the family pet should not be a problem.
An interesting note - toxoplasmosis can also be picked up from eating under cooked meat or handling garden soil without wearing gloves.
For more information on toxoplasmosis talk to your vet or family doctor.