Resolved!
I had just downloaded the mame hq mp4 videos from emumovies via the app "Download Artwork and Videos" that was installed after I'd installed GameEx. But I checked the setting "Remove fade in from start of video" in the options tab before downloading the set, as I was interested in removing the fade in. I noticed when playing one file just via WMP that there was pretty significant artifacting for just a moment. Anyhow, I swapped the files back out to the ones that fade in, and now all of the preview videos are working fine!
One way that someone could relate this experience to - if you hear about 3 seconds of audio but the video doesn't move, followed by a repeat of the same audio again, and on and on... that'd be what I noticed this problem exhibited.
Point is - Thinking of removing the fade in from the start of videos? Don't.