I think there are some fantasies and/or fetishes that CAN be lived out, and others that cannot. It really depends on the situation, your significant other, etc.
Having kids makes things harder and generally, more of the burden and exhaustion falls on women and I suspect that is at least part of the decreased sex drive, although I am not an expert and am saying that is a "general" not a definite.
When I was married I also found that I tended to talk AT my wife about sex instead of talking WITH her about sex. This was part of my own issues and such at the time, but maybe others fall into the same category as well. Begging or bringing up sex, such as saying "we don't have it enough" isn't the same as trying to get down to what the other person may need or want in order to be more available for sex. Would a weekend at a hotel by themselves with no spouse or kids help? Things like that.
OK, I've gotten a bit off-track on fantasy. I think that there are many times when spouses or the person you are dating are open to some fantasies or ideas. Hell, they may have ideas you wouldn't have thought of as well. But, some things might just be off the table. It is ok to BOTH have fantasies you cannot ever achieve but also to have fantasies that you can share and engage in with the person you love. So I do not think it is an either/or situation.
Anyway, that is my non-educated opinion from the divorcee perspective.