When to return payments:
The main reason for returning payment would be the case when a customer’s check bounces and it has already been included on a deposit. If this is the case, then the GL accounts will be wrong, saying that the bank account balance is the amount of the payment higher and that the customer doesn’t owe anything.
The other reason it may be necessary to return payment, would be if the receivable was made, assigned to an invoice in error, and deposited. In this case the GL accounts would be showing that the amount for the invoice has been paid when it is still due from that customer.

To return payment:

• Open the receivable or prepayment from which payment will be returned.

• Click the button labeled “Returned Payment,” at the top of the receivable.
  

• Choose whether the payment is being returned due to non-sufficient funds or because it was assigned in error.
  
  This will create a duplicate receivable or prepayment with negative amounts, which will show up to be deposited in order to correct the account balances.

• On a receivable click “Post” to apply it. Prepayments will post automatically.
  

Note that the resulting NSF transaction may be posted to Undeposited funds, and will need to be added to a deposit (which will most likely have a negative balance) for it to post to the bank account and show in your bank reconciliation.

See also Receiving/Making Payments Tips and Tricks.