The WP admin GUI reference was just a note to the fact that unless you also do a proper uninstall of the plugin in WP, and then remove the plugin files, you might end up with a broken WP because it’s expecting files to be there but can’t find them.
It’s a bit worrying to hear that you think using snapshots to version control a website is better or easier than using Git. This is exactly what Git was made for. It’s the right tool for the job.
With WP you must be careful to keep the database and the files in sync, otherwise you’ll have more trouble than what’s pleasant (even though it can usually be fixed).
If at the time that you want to restore to a good state you are also prepared to lose whatever data was added to the site after your last good “snapshot”, then you will have to restore that snapshot AND restore the database.
If you don’t want to lose data that was added to the site since you installed the plugin you want to get rid of, you’ll need to use e.g. Git to undo the add plugin commit (or some other method, like the one you’re proposing), but as I said that should be done after uninstalling the plugin using the WP admin tools (just to be on the safe side).
Either way, why not just version control it using proper tool for that. Why would you not version control your site’s source code? I just don’t get it
PS: Snapshots and commits in Git are more or less the same, for the intents and purposes of this discussion.