When you introduce new mice, it must be done in steps to be successful. You cannot just put them together without having gone through the process. I put the cages side-by-side to get the mice used to each other's presence. The cages should be near, but not close enough that they can take a swipe through the bars. I then swap items between the cages to get them used to the other mouse's scent. Igloos work well as do empty paper towel rolls and plastic tubes. After a few days of swapping items, you can swap entire houses if you want. Put the resident mouse into the new mouse's cage and put the new mouse into the resident mouse's cage. You can next get ready for the meet-and-greet. Use neutral territory so that nobody gets territorial. A bath tub works well as does a couch with an old sheet tossed over it. Be sure to put down plenty of toys, hide-away homes and other accessories to keep the mice amused. This will also help distract them from fighting. Make sure all of the items you put down are washed and scent-free. There may be squabbles at first whilst the mice are becoming accustomed to each other, but as long as no blood is drawn, the mice will settle down. Increase the length of these meetings each day. You will be able to tell based on how much they squabble how long you will need to have meetings. If the mice get along from the get go, you can arrange for the final step in housing the mice together. Make sure the cage you plan to use for the mice is clean and all the accessories are newly washed. You do not want anything in the cage including the cage to smell like any of the mice. Once the mice are together, there may initially be minor squabbles whilst the mice are establishing the pecking order amongst themselves, but again as long as no blood is drawn, they should be fine.
TFM is an excellent mouse forum. They have a section on introductions. It is as follows:
http://www.thefunmouse.com/info/introducing.cfm
I wrote the above for introducing does. If you have bucks, it would explain why the one mouse is so uneasy. Bucks must NEVER be housed together. Even siblings turn on each other. It is usually sudden and unexpected, and the result is often death or at the very least serious injury. Following are details:
http://www.thefunmouse.com/info/keeping-mice-together.cfm#maleslivetogether
http://forum.thefunmouse.com/viewtopic.php?f=8&t=13274
I am puzzled why you have harvest mice. This implies they are wild. If these are not domestic mice, unfortunately you are not going to be able to tame them.