Author Topic: Scraping  (Read 8659 times)

Grahf

  • Newbie
  • *
  • Posts: 4
    • View Profile
Scraping
« on: February 14, 2019, 09:38:47 AM »
Hi again...

When I go to scrape attract mode displays the following message at the bottom of the screen

"Error Getting Platform List from thegamesdb.net.  Code $1."

Is there anyway to fix this or another scraper I can use?


toothlessgrinn

  • Sr. Member
  • ****
  • Posts: 190
    • View Profile
Re: Scraping
« Reply #1 on: February 19, 2019, 06:55:15 AM »
as far as i know no. but i do understand that raygun is working on the problem. however it does seem to still scrape mame or at least it has for me and a handful of others. and your better off finding rom media packs elsewhere cuz the console rom sets rarely match the media name from the gamsdb site so you get like for say 300 roms about 25 game pics and 275 stock pics or blank (just sayin.its what i did.)

good luck

we came , we saw , we lost LOTS of quarters...

Win 7
Asus mobo amd 64bit 3core
12g ram
4 player. Custom built woody cab
46" plasma screen with 22" marquee

pecosdave

  • Jr. Member
  • **
  • Posts: 22
    • View Profile
Re: Scraping
« Reply #2 on: February 19, 2019, 08:42:41 AM »
I have successfully scraped recently using Skyscraper.

It is not necessarily straight forward, I may be writing a how-to soon, but with a full time job and three kids I don't have an incredible amount of time to do it.  Even with the scraper working I still had to do a few manual interventions (I may be able to avoid that after getting better at it).

I will say I really liked the results when I was done, getting scrapes from (the annoying to navigate, but lots of info) Screen Scraper site gave me videos on things you usually wouldn't expect videos on.

bionictoothpick

  • Sr. Member
  • ****
  • Posts: 320
  • He who laughs lasts.
    • View Profile
Re: Scraping
« Reply #3 on: February 19, 2019, 09:01:58 AM »
I'd like that howto...

pecosdave

  • Jr. Member
  • **
  • Posts: 22
    • View Profile
Re: Scraping
« Reply #4 on: February 21, 2019, 07:32:35 AM »
I've been giving it some thought.

I think I'm going to live-cast my document writing this weekend.  My wife is going out of town today - while I am at work, and I won't see her again until Friday of next week, she's taking my boys with her so I'm actually going to have a chance to do this.

The one I got to work is for my kid.  I'm going to create a profile on my desktop with his login and I'm going to run FreeFileSync to copy what he has onto my system on his profile.  I'm going to setup a sync to sync the document I'm writing up to my web server and I'm going to provide a link, I'm going to have the document sync every five minutes, so that people following along can see it as I go.

I'm going to stream my desktop and I will have a video of me talking running in a corner.  I've done some Internet radio before on both the Black Market Liberty Network and on the Scott Ford the Libertarian Professor shows, but I've never tried something like this while I'm actually working.  My brain tends to shift power away from the language center first and send it to where it's needed when I'm concentrating, so the narration might suck.  Hopefully the video of what I'm doing will suffice when I'm in "verbally stupid mode".

I'm going to have a chat window open - does everyone prefer a Google Hangout or Discord?  I'm going to reserve bandwidth for the stream so the stream won't be there with voice and video, just text.  The video will be where I send it - probably to Bitchute, IPFS.tube or D.tube. I'll return to this thread with details.  I'm in the Central time zone in Texas and I'm going to start on this Saturday morning some time.  No set time, I've got a day off by myself, I may roll out of bed at the crack of noon.

Tonight I'm going to reinstall my main OS.  I'm running Netrunner Linux, which has pretty much been dropped some time back and I'm getting way out of date on it.  I've got KDE Neon running on everything else and I'm going to move my desktop to it.  So what I'm going to do will apply to Debian/Ubuntu derivatives and isn't useless for the rest, but it will take a little more skill to translate to the rest.  I've got /home on it's own drive so I'm going to just pave over my / drive to get going again - what this means:

I'm not going to start the live cast at running Skyscraper - I'm going to start it at compiling Attractmode.  I've never had much luck installing Attractmode from installing pre-compiled .deb files, I've always compiled it from source.  THEN after getting Attractmode working, at least at a cursory level, I'm going to start on Skyscraper, configuring it and getting it ready.  I guess that means if I sync my kids profile over I'm going to have to go ahead and blow-out the config files so I can start over for instructional purposes.

When I'm done with this I'm going to have a cobbled-together document and video.  The document may not make a lot of sense on its own its raw form, but I'll touch it up a little as time goes on - and anyone who wants can reference the video (that I'll link to on the web page I'm setting up) and republish the document or share it back to me and I'll put it back.

Feel special folks - I've just moved things around, my desktop doesn't even have an Ethernet cable running to it yet so I'm either going to get up in the attic and put in another drop after work Friday - and I don't do real work after work on the weekdays often - or I'm going to at least put together a temporary cable to string out across the floor so that I can get this knocked out before putting in the drop.  Either way I'm at least quadrupling the amount of time it would take to get this sync setup without sharing.

pecosdave

  • Jr. Member
  • **
  • Posts: 22
    • View Profile
Re: Scraping
« Reply #5 on: February 23, 2019, 01:24:48 PM »

jasongrimme

  • Newbie
  • *
  • Posts: 1
    • View Profile
Re: Scraping
« Reply #6 on: March 04, 2020, 01:42:14 PM »
Hey Dave,
Is the howto available somewhere else?

Thank you!

Leecher

  • Full Member
  • ***
  • Posts: 56
    • View Profile
Re: Scraping
« Reply #7 on: March 09, 2020, 11:03:15 PM »
I'm not sure if this is the right place to post this.

But this could be a bug in the scrapper. So, I try to scrape Saturday Night Slam Masters for the SNES in thegamesdb, but it just crashes in 0%.

I know you should report bugs in github, but I don't have a github account.