There is no question the some wood can be potentially toxic to some people. The question is only one of degree. This cannot truly be answered. Each individual has different degrees of resistance and some are more prone to allergic reaction. Every day the knowledge of tree biology and chemistry grows, which gives us more information but also more questions. How all this relates to each individual is absolutely impossible to know. We just need to be cautious. Using new woods in a limited way is important especially for two days to see if you have a reaction. Use proper respirators until you know there's no adverse reaction with your body.
Toxicity of wood means different things to different people and here are some of the basic concepts of wood toxicity both natural and man-made:
Natural Chemical Poisons
This refers to the chemicals produced by the tree. These chemicals originate as part of the tree's natural defense system against insects or other animals attacks. They are present in higher concentration in the sap, bark and foliage of the tree and lesser in the trunk. For example, the foliage of a black cherry contains a precursor of cyanide, but when you work with the wood there is very little difficulty. Woods that are naturally oily, even after kiln drying are more likely to cause an allergic reaction. The woods oil can transfer to your skin and enter the bloodstream. The reaction can range from a mild irritant to something being truly toxic and lethal although the latter is very rare.
Natural Physical Poisons
Fine wood dust even when there is no Chemical toxicity can pose a health risk also. Dust acts as an irritant to the skin and to the respiratory tract of people with sensitivity in this area. Western Red Cedar has an extremely small dust particle, which is prone to penetrate deeply into the lungs, and causes severe reaction and asthma sensitive people. Make sure to wear a dust mask and have good extraction equipment working in your shop.
Nature can introduce toxins as the wood rots on the forest floor. Fungal spores and bacteria that invade a rotting tree create toxic wood that it can be toxic if it invades the human body. Mankind is also introduce toxins into would. Every time the wood is chemically treated with products like creosote to prevent rotting and glue such as formaldehyde can be detrimental. When you saw these materials the dust generated introduces the toxins back into the air. Many of these glues admit vapors after the manufacturing process has been completed. Both the dust and the vapor can be very toxic.
The Last Word
Be cautious. Just because something doesn't cause a reaction the first time doesn't mean you will never be allergic to it. Your sensitivities can build with exposure. Allergy prone people should be more cautious in the woods and should limit exposure to sawdust every time you work with it.