As I was told, only pure functions can be compiled. Also, NIntegrate or NDSolve first compiles a function before doing numerical operations. Thus, I would have expected that using pure functions in NIntegrate or NDSolve could be faster than rule based functions.
However, in tests I didn't find such a difference. For example,