How to make a NONperistant ubuntu installation

over the last few years ive noticed i seem to build up a lot of junk on my various machines as i stumble through projects and try to get things working. I'm wondering if there is a way to install ubuntu to a computer much like how the live usb works, only install ubuntu, add some programs, then make any future changes get reset at boot.

So the main question is, how to I install ubuntu and various programs to a computer, then prevent any future changes from being permanent?

This is mostly so I can have a machine that I do all the testing and figure out what needs to happen to get things working, restart said machine to remove all my changes, rinse and repeat. Making the changes not stick seems easier for me since i wont have to manually reinstall the os and reconfigure ssh every time, and easier for the machine since it wont be recopying all of the same files every couple weeks.

Maybe the only way is to modify a live usb disk and just boot to that and skip installing all together, but how would i go about configuring ssh on a live disk? since the majority of the time i wont be able to physically get to the machine.