Ok, have I got this right?
They like to be together at night?
Frank doesn't like being in the cot and prefers the mattress during the night?
Could you all sleep on the floor together for now? Sometimes they just need a little reassurance, then once they settle a bit you can try gently changing your sleeping arrangements again.
You could try sleeping together for a bit, and see if they settle better that way. After a bit, when things seem better, try falling asleep together then transferring them to the cot (that might take several attempts the first few nights, but keep trying and comfort them if they get upset). Then, once they get used to that you can try settling in the cot, all the while gradually reducing your input - so start with the rocking, patting etc to begin with, then when that seems to work well do less, maybe change to music/singing and you just having a hand(s) on them. The next stage is to gradually move away from them until you're out of the room while they're going to sleep.
The key is to do it gradually and always come back to them to comfort them if they're upset becuase pushing changes while they're distressed may only make things worse for you.
I have had some success with the No-Cry Sleep Solution. Maybe try getting a copy at your local library and have a look - she really has some good practical advice on all sorts of situations.

Really hope things get better for you soon