Adding Demoscene Demos to the Database

[keito]

Hi, I love this launcher like many others and I appreciated your hard work and committed development on it.

As I am also a huge demoscene fan and have thousands of demos as can be seen all categorized in the GameBase Demoscene package, and also on numerous sites hosting ADF and DMS images of all demos.

I was wondering would there be a way to incorporate a Demoscene database functionality into FS-UAE? Just think after playing a couple hours of games you might want to relax and you could load up a demo or two, I would love this feature and im sure others would too.

Any thoughts on this?

Keito.

[FrodeSolheim]

Perhaps, I know there are at least some other people who are interested in this (see for example http://eab.abime.net/showthread.php?t=72937).

There are at least two ways to go about this:
- Have an online database (like oagd.net)
- Have an offline database / configuration set which you unpack to your FS-UAE directory (this is of course possible to do already today).

In both cases, it could be useful if the configurations/entries were classified as "Demos", so the launcher could at minimum filter between Games and Demos (etc).

I'm not personally interested in managing a demo database, but I am of course open to make useful adjustments to FS-UAE Launcher if someone steps up to actually create/maintain the database.

[keito]

Yes the database is the key i suppose, I would love to see this happen so i am willing to lend a hand to administrating the db.

The most thorough and very excellent database of amiga demoscene information i know of is: http://janeway.exotica.org.uk/search.php

I wonder if we could scrape data from there, or i could even ask them if we could get backend access to their db, i chat to them on irc too so wouldnt be hard to ask.

Im just thinking aloud im not necescarily saying my idea is the best one, i welcome all and more input

Also thank you for the fast reply FrodeSolheim, its nice to know you devs listen, which is what makes you and your project so special

[Dunny]

Would be nice if you released either added functionality to allow the user to create/modify their own local database, or release tools to do it separately from FS-UAE.

D.

[keito]

I agree, having the option to manage a personal local db would also be a good idea, and one that might work if trying to make a public db fails.

I also use AmigaForever interface for making my own local db, but I really dont like how it handles window resizing at all, it breaks the flow of watching demos, so that's one reason why i would love this feature in FS-UAE as it is far the superior emulator and interface.

[jbl007]

Originally Posted by FrodeSolheim

I'm not personally interested in managing a demo database, but I am of course open to make useful adjustments to FS-UAE Launcher if someone steps up to actually create/maintain the database.

There is something... As far as I can see the auto-download-feature (download_file) is limited to one single file. Many demos/games have more than one floppy image. So if we could have mulit-file-downloads would be an improvement. Perhaps just separate the url with e.g. "|" to preserve compatibility.

I'm about to create an offline demo database. I added one of my favorite demos "by hand" to an empty database to see if/how it works. (pic) With a separate basedir (which could be distributed including screenshots) you can switch between demos/oagd.net easily. But there are still more things to do. Parser to get data from websites (almost done for janeway). Add data to db (haven't even started yet).

[keito]

jbl007: that's interesting, is it relatively simple to create an offline db? I'll have to explore this myself also.

[keito]

ok so i can create/add a demo easily enough, but im not sure how to add the meta data and screenshots?

[jbl007]

Originally Posted by keito

how to add the meta data and screenshots?

You can add metadata to a config file easily. For example "developer = One Nice Guy" or "homepage_url = http://foo.bar". To see the keywords go to oagd.net, choose a game and click "Edit".
Adding screenshots is a little more tricky, because they are usually loaded on demand from the net if they are not in Cache/Images/xxx directory...

In the meantime I made some progress. Now I successfully imported 324 (all???) singe-disk-based "Trackmos" from janeway.exotica.org.uk including screenshots and some metadata into a database. All demos are automatically downloadable. More can be added.
But I really should ask the guys from janeway for permission to (re-)distribute their screenshots before I upload it somewhere.

[keito]

Hi jbl007, thats great work! sounds like you made some awesome progress, ideally i'dlike the ability to add/remove demos from my own db but it would just be good to have your db as a start.

If you want to talk to the janeway guys, hop over to #amigaexotic @ircnet. There you can find me and others connected to janeway website.

I was also thining about the RP9 format for metadata,

http://www.retroplatform.com/kb/15-122

But i think your way is better right now.
Do you think you could teach us lamers how to add demos as you have done at some point? Anyway great work i look forward to more updates

[FrodeSolheim]

Hi jbl007 - Great initiative, I saw you just released an unofficial database

I have some suggestions/thoughts. If it is to be distributed as an .sqlite database, at least FS-UAE Launcher should be able to load this database in addition to the normal database. To avoid "issues"

Fortunately, this work has already been completed in the latest dev versions, and the launcher can now internally handle many separate databases at the same time without getting confused. This feature was really developed to support multi-platform support in the game system, to be able to sync game data for platforms independent of each other (and not needing to sync irrelevant data at all). I can also easily extend that functionality to allow loading data from "local user databases" also...

But one question is still, is using an .sqlite database the best way to do this? The main reason for having the database for oagd.net is for efficient syncing of changes from the server, which does not apply to this case. Another reason for using the database system is to allow the Launcher to scan/find only the games *you have*. For a demo database where all demos are always available, this functionality isn't needed either.

So another option is to simply use a set of normal .fs-uae files. You can for example have a directory structure like this:

Documents/FS-UAE/Demos/F/Foobar (1991, Demo Group).fs-uae

(FS-UAE will automatically find these .fs-uae files as well, they do not have to be placed inside the Configurations directory).

floppy_drive_0 can be set to a http URL, and the floppy will then be downloaded on demand. floppy_drive_1 can be set to another URL, and then downloading multiple distinct files will just work out of the box. I *think* I coded it so you can reference files within zip archives as well: http://host/path/to/zip#filename-within-zip.

Another advantage is that you can easily create a git repository containing the Demos directory structure, making collaboration easier (compared to the offline .sqlite database at least).

It should also be quite easy to create a script which runs through those .fs-uae files, predownloads the disk images, modify the .fs-uae files to point to local files instead for a complete offline database bundle including disk images. (floppy_drive_x paths can be prefixed with $CONFIG/ where $CONFIG will be expanded to the directory containing the .fs-uae configuration file in question).

[jbl007]

I agree, a sqlite db is not the a good solution, not flexible enough. It was just fun to make one.
It can be converted to .fs-uae files without problems. I already tested a collection of plain config files. Looks good so far. The only thing I miss: (not counting the cute green arrow, hehe) you can't add a variant_notice to the status bar.
It's great to see the launcher will support multiple platforms/sources some day. How do you plan to switch between them? Via entry in config (platform=xxx) or distinguish between different databases/directories? Currently the launcher doesn't like it if platform != Amiga.

Originally Posted by FrodeSolheim

floppy_drive_0 can be set to a http URL, and the floppy will then be downloaded on demand.

Thats a very useful feature. Didn't know that before. Unfortunatly it seems broken in the current dev-version. It breaks at GameDatabase().get_license_code_for_url() It needs to return None or something usefull to work. Another thing is, configparser goes crazy if you add quoted urls, because it wants to deal with the "%" somehow.

I like the git(hub) idea, but I doubt people will contribute.

[FrodeSolheim]

Originally Posted by jbl007

The only thing I miss: (not counting the cute green arrow, hehe) you can't add a variant_notice to the status bar.

The option names are transformed by FS-UAE Launcher to some internal option names (but you *can* set these in .fs-uae config file, it just isn't officially supported ):

x_game_notice = Message 1
x_variant_notice = Message 2
x_variant_warning = Message 3
x_variant_error = Message 4

Originally Posted by jbl007

It's great to see the launcher will support multiple platforms/sources some day. How do you plan to switch between them? Via entry in config (platform=xxx) or distinguish between different databases/directories? Currently the launcher doesn't like it if platform != Amiga.

Correct, platform can be set to for example super-nintendo. But note that I don't really intend FS-UAE Launcher itself to become a huge everything-in-one-monster. But instead, the goal is to have an "FS Game Center" (FS-UAE Arcade with more platforms). A side-effect is that FS-UAE Launcher also understands (some) multi-platform stuff since these projects shares library code

Originally Posted by jbl007

Thats a very useful feature. Didn't know that before. Unfortunatly it seems broken in the current dev-version. It breaks at GameDatabase().get_license_code_for_url() It needs to return None or something usefull to work. Another thing is, configparser goes crazy if you add quoted urls, because it wants to deal with the "%" somehow.

Thanks for the bug report, I've fixed both issues for the next release!

Originally Posted by jbl007

I like the git(hub) idea, but I doubt people will contribute.

I agree, git(hub) wouldn't help most people. But having configs in files would at least make it easier for people to contribute fixes by mailing updated configs (or something), and git would still be helpful to track changes, even if committed by someone.

[jarre]

what happened to janeway exotica, can't reach this site for the last couple of days...gives me


http://janeway.exotica.org.uk/search.php

2a00:1098:0:80:1000:f:0:3 failed.

(101) Network is unreachable

[StingRay]

Originally Posted by jarre

what happened to janeway exotica, can't reach this site for the last couple of days...gives me


http://janeway.exotica.org.uk/search.php

2a00:1098:0:80:1000:f:0:3 failed.

(101) Network is unreachable

Kestra has been dead for a few days. I'll ask Zeg once I get hold of him on IRC what's happening.

[jarre]

Originally Posted by StingRay

Kestra has been dead for a few days. I'll ask Zeg once I get hold of him on IRC what's happening.

OK, too bad, i like that site very much...

[jarre]

it seems all is up and running again, even seems its faster than before

[germandroide]

Originally Posted by FrodeSolheim

Correct, platform can be set to for example super-nintendo. But note that I don't really intend FS-UAE Launcher itself to become a huge everything-in-one-monster. But instead, the goal is to have an "FS Game Center" (FS-UAE Arcade with more platforms). A side-effect is that FS-UAE Launcher also understands (some) multi-platform stuff since these projects shares library code
.

I think it's a great idea, there is EmulationStation that can integrate emulators of any platform and FS-UAE can be integrated as well. The problem is that only Retroarch's Core UAE can launch Amiga content and use the graphic filters to adapt to the Bezels.

Is it possible that Fsuae can be integrated in Retroarch to be able to have in this way solved the scrapping of contents and the integration with other platforms?

[triki1]

Originally Posted by germandroide

I think it's a great idea, there is EmulationStation that can integrate emulators of any platform and FS-UAE can be integrated as well. The problem is that only Retroarch's Core UAE can launch Amiga content and use the graphic filters to adapt to the Bezels.

Is it possible that Fsuae can be integrated in Retroarch to be able to have in this way solved the scrapping of contents and the integration with other platforms?

https://retropie.org.uk/forum/topic/...rk-with-fs-uae
https://wintermute0110.github.io/LKE...onStation.html
That work good with whdloadgames