26 October 2013, 19:49 | #1 | |
Registered User
Join Date: Mar 2013
Location: Leipzig/Germany
Posts: 468
|
Local access to OAGD.net configurations
Quote:
How is the data encoded inside the local db now? I'm asking, because I'm about to write a command line launcher for linux which also has (or had ) access to the local db. BTW: I would be really awesome to query the online db from "outside". For example ask the server: Hey, I have a disk image with SHA1 yxz, gimme a config for this game and the SHA1s of the corresponding disk images. That's I was trying to do with the local db. Last edited by FrodeSolheim; 26 October 2013 at 20:33. |
|
26 October 2013, 20:13 | #2 | |
FS-UAE Developer
Join Date: Dec 2011
Location: Førde, Norway
Age: 43
Posts: 4,043
|
Not less open, just more compact encoding in the launcher database
Quote:
Code:
cursor.execute("SELECT data FROM game WHERE id = ?", (game_id,)) data = zlib.decompress(cursor.fetchone()[0]) doc = json.loads(data) next_parent_uuid = doc.get("parent_uuid", "") while next_parent_uuid: cursor.execute( "SELECT data FROM game WHERE uuid = ?", (sqlite3.Binary( unhexlify(next_parent_uuid.replace("-", ""))),)) data = zlib.decompress(cursor.fetchone()[0]) next_doc = json.loads(data) next_parent_uuid = next_doc.get("parent_uuid", "") # let child doc overwrite and append values to parent doc next_doc.update(doc) doc = next_doc |
|
26 October 2013, 21:03 | #3 |
Registered User
Join Date: Mar 2013
Location: Leipzig/Germany
Posts: 468
|
Thank you very much for your quick reply, Frode. Your help is greatly appreciated.
Ok, gzip + json. I thought I also tried to decompress using gzip but it did not work for some reason. I'll give it another try... But the basic problem still exists: you can't search by sha1 easily. You need to decompress/decode at runtime > 12000 entries. I think this will be very slow. I will do some tests. I did something like this before, btw.: Code:
search = dohash(filename) searchname = ('%' + search + '%', 'file_list') conn = sqlite3.connect(db) c = conn.cursor() c.execute('SELECT game,value FROM value WHERE value LIKE ? AND name IS ?', searchname) searchgameno, namefetched = c.fetchone() |
26 October 2013, 21:06 | #4 |
FS-UAE Developer
Join Date: Dec 2011
Location: Førde, Norway
Age: 43
Posts: 4,043
|
No, zlib, not gzip. It is basically the same compression algorithm but not the same file format.
Regarding performance, you will probably want to scan through the entries once and create a cache/database with a sha1 -> game_uuid mapping |
26 October 2013, 21:23 | #5 | ||
Registered User
Join Date: Mar 2013
Location: Leipzig/Germany
Posts: 468
|
Quote:
Quote:
I just made a single file python3 launcher script, which shortens some --really-long-command-line=options, makes a little config file handling and automates .zip/.lha extraction. I didn't want to rewrite the fs-uae-launcher with databases, config files and so on. With my limited python skills it would take ages. |
||
06 November 2013, 20:47 | #6 |
Registered User
Join Date: Mar 2013
Location: Leipzig/Germany
Posts: 468
|
Actually is was quite easy to get a sha1->variant-list dictionary out of the db based on your example code. And it only takes 0.5s on my i5, so caching isn't really needed. But I did cache the sha1's of the users disk images, but that's another story...
I hope the db-format will not be changed in the near future. I almost got my one-click-launcher at a usable state. I'll push it on github and post the link here. |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Thread Tools | |
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
oagd game images | DaveMB | support.FS-UAE | 2 | 06 October 2013 23:51 |
OAGD No Screenshots on 1024x600 Display ? | liveevil | support.FS-UAE | 1 | 06 May 2013 00:40 |
Wireless net access with Minimig ? | lolafg | support.Other | 2 | 26 January 2011 19:26 |
Configurations? | Worf | support.WinUAE | 12 | 02 March 2009 22:35 |
configurations | Unregistered | support.WinUAE | 3 | 29 August 2002 23:01 |
|
|