Brown says Lee shouldn't have taken Mirkarimi's pay away