Attract-Mode Support Forum

Attract-Mode Support => General => Topic started by: na1337 on July 26, 2016, 02:18:23 PM

Title: "generate Collection/Rom List" takes VERY long
Post by: na1337 on July 26, 2016, 02:18:23 PM
Hi everybody,
I'm using my Rasperry Pi 3 with Attract Mode and everything is coming together nicely. Thanks for the tutorials and help here on the forum that I've been silently reading.
One question: I copied for testing purposes 40 Atari2600 roms to my Attract Mode. If I go to "generate Collection/Rom List" it takes about 2 mins. Is that normal? I'm scared if I have like 2000 MAME roms that it takes two days ....?!
Is that normal? Am I missing something?

Thanks for your help!!
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: progets on July 27, 2016, 11:53:31 AM
I have found that the speed depends on thegamedb.net website. Sometimes it's fast and sometimes it's slow but this issue is also true with other frontends that use thegamesdb.net such as EmulationStation. Also, MAME games don't use thegamedb.net they use local files like mame.xml and controls.ini so the list generation should be very fast if you have it setup properly.
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: na1337 on August 07, 2016, 03:47:24 AM
thanks, I just dared to compile the mame list, everything went pretty fast.
No idea why, but Atari 2600 takes ages.  ???
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: Leecher on July 25, 2021, 12:07:01 AM
I'm gonna revive this just to comment.

I also too think that "generate Collection/Rom List" for masterlists are too slow. It should just combined all of the already generated game list of the selected emulators. It doesn't need to go to thegamedb.net all over again.
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: progets on July 25, 2021, 12:19:55 AM
I haven't tested but it should use your already scraped artwork so there is no need to "scrape" from anywhere. It's going to use the info and art from the other emulators anyways.

Turn off the auto scrape of all the artwork before you generate these lists. I think your (also default) config scrapes for any games it adds. By turning this off it should quickly build the list and not scrape for anything and use the artwork you already have for each emulator.

You can turn the auto scrape back on once you're done, if you want.
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: Leecher on July 25, 2021, 01:44:45 AM
@progets
I'm actually talking about the "generate Collection/Rom List" of my masterlist. I didn't realize that "generate Collection/Rom List" is also the title for generating the game list per emulator. I hope that didn't confuse you.

Where is this "auto scrape" option? I can't seem to find it.

Also, I don't think  "generate Collection/Rom List" for my masterlist is scraping images. I think it's just getting the games' info from thegamesdb.net instead of just copying from the emulators generated list.
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: progets on July 25, 2021, 02:38:41 AM
If you're talking about a "masterlist" in some image you got somewhere I can't help you because I don't know what you're using. I am the the author of the masterlist on this forum which can be seen here http://forum.attractmode.org/index.php?topic=1165.msg8455#msg8455 (http://forum.attractmode.org/index.php?topic=1165.msg8455#msg8455)

The settings I'm talking about are in attract-mode under Configuration-->Scrape

Anytime you create a romlist with the settings under Configuration-->Scrape are "on/yes" it will scrape whether or not you have the metadata or artwork already.

Try it and see if it works.
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: Leecher on July 25, 2021, 05:39:31 PM
@progets
I'm talking about the masterlist that comes with 2.6.1 I'm not using any separate plug-ins. I'm assuming that it's getting the info again because of the time it takes to create the master list.

Also, I don't have "Configuration-->Scrape-->(auto-scrape or re-scrape)". I have "Configure-->Scraper-->(see attached pic)"
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: progets on July 25, 2021, 08:37:24 PM
Your image shows that you have some scraper setting turned on. "Yes" = on, "No" = off. With these set to "Yes" attract-mode will scrape when you generate romlists.

I'm suggesting that you try turning those off before creating a collection list and see if helps.
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: zpaolo11x on July 25, 2021, 11:26:10 PM
As far as I remember, when you create a "romlist of romlists" metadata is not transferred from the original romlists, so some form of scraping for metadata is still needed. I'll check better later.
The main problem with such kind of romlists is that the collection is not bounded to the original lists, so no favorites or tags are transferred back and forth between the two
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: Leecher on July 25, 2021, 11:59:32 PM
@progets
Disabled the image scraper. And it's still slow... Turn it back on again because I need the images. Just like zpaolo11x said, there's "some form of scraping for metadata is still needed".

It should just copy the details from the already generated romlists, it doesn't have to scrape meta data all over again when creating a "romlist of romlists"

Title: Re: "generate Collection/Rom List" takes VERY long
Post by: progets on July 26, 2021, 12:42:16 AM
I haven't tested but I pretty certain that all of the data is pulled from the emulator.cfg files when using a combined list. When I did the masterlist list linked above, I don't think I needed duplicate artwork. Also, for a combined romlist to work it would need the correct command line variables that are only located in each emulator.cfg.

Scraping is the only thing I can think of that would take a long time. Maybe you can temporarily remove the info_source fields from the emulator.cfg files you're combining. This would give it nowhere to scrape.

You jumped on an already existing thread and haven't told us what you're trying to accomplish. You might be better off just using the command line to combine your lists which takes no time all.
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: progets on July 26, 2021, 12:59:31 AM
The main problem with such kind of romlists is that the collection is not bounded to the original lists, so no favorites or tags are transferred back and forth between the two

This is true which I say in the above linked thread. The key here is to use just the mater romlist and create all favorites and tags and then use filters for everything else. This overcomes this limitation. Other have also written simple scripts to overcome this as well.
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: Leecher on July 26, 2021, 01:02:33 AM
Quote
You jumped on an already existing thread and haven't told us what you're trying to accomplish. You might be better off just using the command line to combine your lists which takes no time all.

@progets
Sorry if I angered you. I just want to comment in this thread because of that minor issue. I think it's slow, but it's done in 1 minute and 15 seconds (109 games), so it's not that bad. That's why I didn't create a new thread for this. I can use cmd commands, but I'm not studying AM's source code to do it manually. I just want to use AM, instead of creating my own launcher. Sorry again.
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: progets on July 26, 2021, 01:21:39 AM
I'm not angry and there is no need for an apology. I just don't know what you're trying to do since the thread subject isn't yours.

To be honest I think a romlist done in a couple of minutes is pretty good.
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: Leecher on July 26, 2021, 06:06:46 AM
@progets
So, do you have the command-line for combining my romlists, so I don't have to use the "generate Collection/Rom List" for my "romlist of romlists"?

Because if it's 2mins for 100games, it's probably gonna be 10mins for 500 games. And I'm still currently adding games.
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: progets on July 26, 2021, 01:45:58 PM
Quote
ROMLISTS: Collection/Rom lists are saved in the "romlist" subdirectory of your Attract-Mode config directory. Each list is a semi-colon delimited text file that can be edited by most common spreadsheet programs (be sure to load it as "Text CSV"). The file has one game entry per line, with the very first line of the file specifying what each column represents.

In addition to the romlist generation function available in config mode, Attract-Mode can generate a single romlist containing roms for multiple emulators from the command line using the following command:

   attract --build-romlist <emulator names...>

Found here https://github.com/mickelson/attract/blob/v2.6.1/Readme.md (https://github.com/mickelson/attract/blob/v2.6.1/Readme.md)
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: Leecher on July 26, 2021, 03:41:09 PM
@progets
Quote
attract --build-romlist <emulator names...>

How can I use this? Let's say I have three emulators; like this?
Quote
attract --build-romlist "Nintendo Entertainment System", "Super Nintendo Entertainment System", "Game Boy Advance"

Where do I put the name of the output file, "All Systems"?
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: progets on July 26, 2021, 08:50:21 PM
Open the command line. Change to your Attracmode directory. Run the command.

Code: [Select]
attract-console.exe -i ".\romlists\Nintendo Entertainment System.txt" -i ".\romlists\Super Nintendo Entertainment System.txt" -i ".\romlists\Game Boy Advance.txt" -o "All Systems"
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: Leecher on July 27, 2021, 02:06:57 AM
@progets
Thanks progets. That is much, much better. It only took a couple of seconds.
Title: Re: "generate Collection/Rom List" takes VERY long
Post by: akafox on July 30, 2021, 10:51:24 AM
I will toos my two chicken wings in here...this is mainly for people that find this later.

As was stated before if you create a list against the gamedb.net it can take a long time...sometimes "forever" (I left mine running one night to make a list of megadrive games...woke up..it is till wasn't done) It all depends on the what ever network issues per say I guess. I tried the same list a day later (26 hours) and I could get it in about a minute.

You can scrape locally and it is FAST..but it's only a half tailed list....

You CAN use a player.ini and a category.ini ect. for making console lists and is it FAST (it's all local). Arcade roms have a standard to make them easy to "scrape" from an xml file but the problem is the naming format for console roms are not. Your roms may not be named exactly as mine are therefore it doesn't pick up those roms or even work at all. After you scrape everything you can make your own custom "information files" and then use them as you see fit however. It's tedious work at best but I can build a fully filled out list without any internet connection at all. But most won't take the time or trouble.

Best thing to do is to backup your lists after you have them built. After that you can pull info from them or even re-use them if needed to save time. Remember the lists are just plain text files separated by semi-colons..so they are TINY and easy to store.