I had two thoughts. One was ... You really only need an angle and a distance. The distance is the amount you expand and the angle can be derived from the x/y coordinates.

http://jsfiddle.net/uLu7v/38/

first you calculate the angle:

var angle = Math.atan2(hoveredY - circleY, hoveredX - circleX);

Then you calculate the distance they move:

var topMove = ((expand /2 ) * Math.sin(angle)); // sin for Y
var leftMove = ((expand /2 ) * Math.cos(angle)); // cos for X

then use jQuery's built in animate +=:

$this.animate( { 
    "left": "-=" + leftMove + "px",
    "top":  "-=" + topMove + "px"
}, 75 );

Tada!

I had two thoughts. One was ... You really only need an angle and a distance. The distance is the amount you expand and the angle can be derived from the x/y coordinates.

http://jsfiddle.net/uLu7v/38/

first you calculate the angle:

var angle = Math.atan2(hoveredY - circleY, hoveredX - circleX);

Then you calculate the distance they move:

var topMove = ((expand /2 ) * Math.sin(angle)); // sin for Y
var leftMove = ((expand /2 ) * Math.cos(angle)); // cos for X

then use jQuery's built in animate +=:

$this.animate( { 
    "left": "-=" + leftMove + "px",
    "top":  "-=" + topMove + "px"
}, 75 );

Tada!

Edit:- A couple of improvements in the code not supplied in the question:

In function inCircle() change the return statement to return (mouseDistance <= radius); Using the ternary operator to return a boolean? ... I'm sure it used to return some other values right?

In $( ".circle" ).click( :

if(!$( this ).data( "clicked" ) && inCircle( $( this ), event.pageX, event.pageY ) ) {
        
    $( this ).data( "clicked", true );
    setLocations( this, 200, event );

} else {
    
    resetLocations();
    $( this ).data( "clicked", false );
    
};

Makes it cleaner but not a real improvement otherwise, more of a preference.

In function setLocations() your circle parameter conflicts with the local circle variable. I would change the parameter to circleElement.