Why am I forced to give back what I have won? (Humankind)

I have conquered 4 English cities, so the leader walks up and shakes hands that he will surrender if I return all of his cities. I have no other choice because I don't have enough military support. There is no way to end the turn until I put everything back. Why should I accept his terms of surrender? It doesn't make any sense