Author Topic: help with scraping on thegamesdb.net  (Read 2341 times)

spinox

  • Newbie
  • *
  • Posts: 3
    • View Profile
help with scraping on thegamesdb.net
« on: December 23, 2021, 02:25:30 PM »
hi everyone, i'm trying to scrape data from thegamesdb.net for a nes game, i'm using default system identifier but nothing happens...
this is what i can find on last_run.log

*** Scraping artwork for: Nintendo NES
 - scraping thegamesdb.net...
 ! Error: None of the configured system identifier(s) are recognized by thegamesdb.net.
 ! Error processing request: SSL connect error (https://api.thegamesdb.net/v1/Games/ByGameName?apikey=035b376cf5769eca89325626f698702fb72bff15ef5d0451e1859bb7b6b854be&name=Adventures%20of%20Lolo&fields=players%2Cpublishers%2Cgenres%2Coverview&filter%5Bplatform%5D=)
 - 1 games not matched on thegamesdb.net: "Adventures of Lolo "
 - thegamesdb.net reports a remaining allowance of: Unknown
*** Scraping done.

first, i tried to use a correct identifier so i managed to fetch platform list at https://api.thegamesdb.net/spec.yaml and the api gives me this response:
(...)
"7": {
        "id": 7,
        "name": "Nintendo Entertainment System (NES)",
        "alias": "nintendo-entertainment-system-nes"
(...)

i try all of them on attract-mode settings but nothing works, same error "None of the configured system identifier(s) are recognized by thegamesdb.net"

actually, if i add 7 after the request shown above, it works in my browser giving me the game's info i was searching for
https://api.thegamesdb.net/v1/Games/ByGameName?apikey=035b376cf5769eca89325626f698702fb72bff15ef5d0451e1859bb7b6b854be&name=Adventures%20of%20Lolo&fields=players%2Cpublishers%2Cgenres%2Coverview&filter%5Bplatform%5D=7

and the SSL error, how can i fix it?
anyone can help me? what i'm doing wrong? thanks, spinox.

progets

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1271
    • View Profile
Re: help with scraping on thegamesdb.net
« Reply #1 on: December 23, 2021, 06:08:36 PM »
Make sure this line is in your emulator.cfg file for NES.

Code: [Select]
system               Nintendo Entertainment System (NES)

spinox

  • Newbie
  • *
  • Posts: 3
    • View Profile
Re: help with scraping on thegamesdb.net
« Reply #2 on: December 23, 2021, 11:29:40 PM »
thanks for your reply. my cfg file contains exactly what you posted.

it seems ok to me... i tried several time and the cfg file reflects the changes i made through the a-m interface so i don't think the problem is here...
same error on last_run.log
"! Error: None of the configured system identifier(s) are recognized by thegamesdb.net."

to add some details i'm using the XP 32-bit version, could someone confirm me that the error doesn't depend of this?

thanks, spinox.

progets

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1271
    • View Profile
Re: help with scraping on thegamesdb.net
« Reply #3 on: December 24, 2021, 01:42:26 AM »
XP is old. I understand why people use it, mostly old hardware.

I don't know what your issue is, I don't have an XP build to test against. I can try to provide some options for you to troubleshoot your issue.

1. thegamesdb.net is notoriously unreliable. It's possible it will work fine tomorrow or a week from now.
2. since XP hasn't gotten updates in years you might have expired SSL certificates that need to be manually updated.
3. there is a special XP build on the forum. I don't know how it's different but if someone made it, there's a reason. I don't think this will help your particular issue but it's worth pointing out.
4. thegamesdb.net changed the way they work a couple of years back. If you're using an old version of attactmode it won't connect to thegamesdb.net properly. If this is the case track down the special XP version mentioned in #3.

Possible solutions/test options...
1. copy your attractmode build to a modern OS (assuming you have a newer OS) and see if you have scrape issues there.
2. use third party scraping. there are free options out there.
3. post your emulator.cfg file here so we can look at it and make certain there aren't any issues.
4. run linux on your hardware (more modern versions for old hardware than XP)
5. replace hardware with a cheap (under $50) SBC like a Raspberry Pi

« Last Edit: December 24, 2021, 01:48:30 AM by progets »

spinox

  • Newbie
  • *
  • Posts: 3
    • View Profile
Re: help with scraping on thegamesdb.net
« Reply #4 on: December 24, 2021, 04:01:59 AM »
hi, thanks for your advices...

- i think my ssl certificates are ok, i updated a while ago to TLS 1.x to make email clients work;
- if i scrape with the requests tha a-m generate on my xp web browers they works;
- i'll search for the xp version you told me;
- i'll search for an alternative scraper;
- my cfg file is below.

even if my main computer is also xp (!), what i'm actually trying to do is to recycle a netbook with xp on and a usb controller as a gift. i've tried some all-in-one linux distributions as batocera but they are too slow while a-m with emulators runs fine.

thanks again, spinox.