The passage appears near the end of Book 2 of De Genesi ad litteram (The Literal Meaning of Genesis). There's a relatively recent (1982) English translation by J.H. Taylor, S.J., in the series Ancient Christian Writers (No. 41), published by the Newman Press. Unfortunately, Taylor modernizes "mathematician" into "astrologer." Here's an excerpt:
37. Hence, we must admit that when astrologers speak the truth, they are speaking by a mysterious instinct that moves a man's mind without his knowing it. When this happens for the purpose of deceiving men, it is the work of evil spirits. To these spirits some knowledge of the truth about the temporal order has been granted, partly by reason of their keen and subtle senses, since they possess bodies of a much more subtle nature than ours, partly because of their shrewdness due to the experience they have had over the long ages they have lived, partly because the good angels reveal to them what they themselves have learnt from Almighty God, at the command of Him who distributes man's merits by the right principles of His hidden justice. But sometimes these wicked spirits also feign the power of divination and foretell what they themselves intend to do. Hence, a devout Christian must avoid astrologers and all impious soothsayers, especially when they tell the truth, for fear of leading his soul into error by consorting with demons and entangling himself with the bonds of such association.