I made a mistake with the rubber plugs once many years ago. Forgot where they went and thought no way they would put a plug over the pilot jet. And it ran like crap until it was revved out, but it also didn't idle good was only firing on one cylinder, way too much fuel. So if it is idling fine I don't think it is the rubber plugs.
Another problem I had once many years ago, was sled idled fine, was a little burbly off idle, but got really bad gas milage. So not thinking right and riding in a new area I think the pilot circuit is rich and change pilots, pilot screw, still runs bad. Then for some reason I look at the choke lever and notice it is not seating all the way and play with the cable and no improvement, so pull the chokes out of the tubes and one had a broken spring as well as some grunge. Found a spring that would work temporarily and was good to go.
I would be inclined to go with Mr Sleds thought and have has this problem many times with Phazers through the years. Check the sliders in the primary clutch. For some reason they will get tight, they get a little dirty and scratched up and do not slide smooth and the clutch starts sticking. I have taken them out on a trip when new parts weren't available and filed them down for a quick fix. Don't know why but they seem to swell up and the clutch will hang. First time I ever noticed this, was running the sled on a stand and revved it up, and the clutch would hang and them snap shut with a bang on deceleration. Also seem to have to replace the sliders every couple of years.