hi.
I'm so sorry you are in this position, especially as you thought that you'd had everything settled with regards to visitation. We don't know your entire story, so any suggestions or judgments are made on what limited information we do have.
I don't have kids yet, pregnant with my first at the moment, but I really feel for you and the extremely unfortunate situation your ex husband has put you in. Your fiance is right, this is a decision only you can make.
A couple of good points have been made. Since your fiance's finances and accommodation will be used for your visa, can those be used as proof of support for custody? Obviously, your daughter would not be making the trip along as there are airline restrictions regarding how old kids need to be for unaccompanied international travel, if your ex is concerned that she would be doing the trips alone, can this information be gathered to reassure him? Do you have family near your ex who you could take a long visit with and see your daughter then if you do move.
I'd look for lawyers, both one in the states and one in the UK that specializes in international custody disputes. They will be the best people to offer suggestions as to how to progress and how any actions you take may be viewed by the courts. And unfortunately this will most like have to go to the courts. They may see this as you moving to be able to provide better for your daughter and not count that against you, but that is something only the experts will be able to tell you.
I would ask your ex one thing, as it sounds like he is limiting your access even now, if you did plan to stay, would he allow you more or will he put further restrictions on you visiting your daughter? if he will continue to make it difficult to see her, how will you being in england make any difference to how often you are allowed visitation. If that makes sense.
I'm assuming you haven't applied for your visa yet, so I'd definitely seek some expert legal advice before you decide.
Good luck.