A study that analyzed sperm samples from nearly 43, men from around the world over a few decades found that sperm counts have dropped by more than half over the last 40 years. Although the definitive cause is unknown and additional research is needed to find the cause of the drastic decline in sperm counts , researchers have identified some habits and health conditions that may cause a decrease in sperm production. Here are some things that can negatively impact your sperm count and sperm quality. The Cleveland Clinic notes that 5-alpha-reductase inhibitors used to treat prostate enlargement and hair loss such as finasteride, dutasteride, and propecia, can reduce one's volume of semen and number of sperm within their semen. In some cases, this drop in semen production is temporary or reversible.