Damaged Plaster..how Much Do I Need To Remove

[Lxb]

My husband and I bought a 102 year old duplex that we converted into a single family. Unfortunately, none of the trim in the house matches, so we decided to replace all of it. This was huge mistake. Most of the trim was put in with multiple, probably hand-made nails, which made it extremely difficult to remove. Now we dealing with cracked and broken plaster. My husband thinks we can just patched and tape the cracks/holes and skim coat the walls since most of the damage will be covered with new trim. I think we have to remove all the damaged areas down to the lath and then apply new plaster. How should we proceed? Thank for the help!