ScoreHero
Home | Forum | Wiki
Inbox [ Login ]Inbox [ Login ]
SearchSearch MemberlistMemberlist
ProfileProfile Log inLog in
GH6/GH5/BH .NOTE File Format
Goto page Previous  1, 2, 3, 4, 5, 6, 7  Next
 
Post new topic   Reply to topic    ScoreHero Forum Index -> Software
View previous topic :: View next topic  
Author Message
raynebc  





Joined: 16 Jun 2008
Posts: 983

PostPosted: Mon Apr 02, 2012 7:20 pm    Post subject: Reply with quote

I did find that eventually there eventually appeared to be individual lyric entries in the PAK file, but I didn't recognize the format they're in. For example:
Code:
0D 0A 65 39 65 62 36 31 66 36 20 22 5C 4C 64 61 72 6B 22

Those aren't all printable characters, but here's what that generally looks like in ASCII
Code:
  e9eb61f6 "\Ldark"

Each lyric seems to end with a quote mark character, but that's the only trend I see here. These individual lyrics don't appear to be in order, and I don't see a timestamp like what we'd expect from GH5 files. Do you have any suggestions where I go from here? Is there a file spec for the QS format someplace?
Back to top
View user's profile Send private message
codemann8  





Joined: 10 Apr 2007
Posts: 1045
Location: Oshkosh, WI

PostPosted: Mon Apr 02, 2012 7:43 pm    Post subject: Reply with quote

raynebc wrote:
I did find that eventually there eventually appeared to be individual lyric entries in the PAK file, but I didn't recognize the format they're in. For example:
Code:
0D 0A 65 39 65 62 36 31 66 36 20 22 5C 4C 64 61 72 6B 22

Those aren't all printable characters, but here's what that generally looks like in ASCII
Code:
  e9eb61f6 "\Ldark"

Each lyric seems to end with a quote mark character, but that's the only trend I see here. These individual lyrics don't appear to be in order, and I don't see a timestamp like what we'd expect from GH5 files. Do you have any suggestions where I go from here? Is there a file spec for the QS format someplace?
yep, qs files are text files, not binary files...the e9eb61f6 is the check sum and 'dark' is the lyric...the \L is on all the lyrics for some reason, just ignore it

As far as timestamps, looks like the 'On' lyric starts at 0xD8A4 ms, this is in the QB file
_________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs



Back to top
View user's profile Wiki User Page Send private message Send e-mail Visit poster's website XBL Gamertag: codemann8 PSN Name: codemann8
raynebc  





Joined: 16 Jun 2008
Posts: 983

PostPosted: Mon Apr 02, 2012 9:02 pm    Post subject: Reply with quote

Doing a search for the first instance of the timestamp (0x0000D8A4) that occurs after the "hotelcalifornia_lyrics" header, I run into 40 byte entries for what appears to be each of the vocal notes. Here's the first one:
Code:
00 00 D8 A4 00 02 49 14 00 01 1C 00 C4 74 58 38 D8 C7 02 4A 00 00 00 00 00 00 01 00 00 02 49 2C 00 01 01 00 90 6B 67 BA

Here's the lyric entry for "on":
Code:
0D 0A 65 31 62 30 39 33 65 32 20 22 5C 4C 3D 6F 6E 22

which looks like this in ASCII:
Code:
  e1b093e2 "\L=on"

Each of the lyric entries does appear to be prefixed with 0x0D0A, but 0xe1b093e2 does not occur within the 40 byte entry. It does exist elsewhere in the PAK file though, but I don't see the timestamp nearby. I am definitely missing some of the pieces to this puzzle.
Back to top
View user's profile Send private message
codemann8  





Joined: 10 Apr 2007
Posts: 1045
Location: Oshkosh, WI

PostPosted: Mon Apr 02, 2012 9:23 pm    Post subject: Reply with quote

raynebc wrote:
Doing a search for the first instance of the timestamp (0x0000D8A4) that occurs after the "hotelcalifornia_lyrics" header, I run into 40 byte entries for what appears to be each of the vocal notes. Here's the first one:
Code:
00 00 D8 A4 00 02 49 14 00 01 1C 00 C4 74 58 38 D8 C7 02 4A 00 00 00 00 00 00 01 00 00 02 49 2C 00 01 01 00 90 6B 67 BA

Here's the lyric entry for "on":
Code:
0D 0A 65 31 62 30 39 33 65 32 20 22 5C 4C 3D 6F 6E 22

which looks like this in ASCII:
Code:
  e1b093e2 "\L=on"

Each of the lyric entries does appear to be prefixed with 0x0D0A, but 0xe1b093e2 does not occur within the 40 byte entry. It does exist elsewhere in the PAK file though, but I don't see the timestamp nearby. I am definitely missing some of the pieces to this puzzle.
It's a text file, not a binary file, 0x0D0A represents the newline and carriage return 0x65 is 'e', 0x31 is '1', etc

And the first record in the lyrics section is (at 0x248FC):
Code:
00 00 01 00 
00 02 49 04 --offset to data (next byte)
    00 01 01 00
        90 6B 67 BA --some checksum (looks same for every lyric)
        00 00 D8 A4 --start of lyric (ms)
        00 02 49 14 --offset to the next data (next byte)
            00 01 1C 00 
                C4 74 58 38 --some checksum (looks same for every lyric)
                D8 C7 02 4A --checksum of lyric text (qs file)
                00 00 00 00

_________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs



Back to top
View user's profile Wiki User Page Send private message Send e-mail Visit poster's website XBL Gamertag: codemann8 PSN Name: codemann8
raynebc  





Joined: 16 Jun 2008
Posts: 983

PostPosted: Mon Apr 02, 2012 11:14 pm    Post subject: Reply with quote

I'm glad you're willing to guide the blind, but I still can't find what 0xD8C7024A is supposed to be a checksum for. It's not the checksum for the text on, On, \L=on, "\L=on" or even the entire line of text e1b093e2 "\L=on". That checksum is also not given anywhere else within the PAK file so it must be generated against text instead of being stored as a checksum someplace.
Back to top
View user's profile Send private message
skylerinman  





Joined: 03 Oct 2007
Posts: 328
Location: Gainesville, FL

PostPosted: Tue Apr 03, 2012 2:46 pm    Post subject: Reply with quote

codemann8 wrote:
I don't know what you mean, this thread is purely an informational thread, not looking for anyone to test anything


I'm aware, but there's always the possibility.
_________________
PS2 setup: Fat PS2 with Free MCboot v1.8, with a 120gb HD.
myvidsomg
Desertman123 is a God, he knows his shit.
dclapp wrote:
Fork1 wrote:
I quit. This team is Deadhead6391s now.
you quit? so no more hacking YT accounts then?
Back to top
View user's profile Send private message Send e-mail Visit poster's website XBL Gamertag: skylerinmanSH
codemann8  





Joined: 10 Apr 2007
Posts: 1045
Location: Oshkosh, WI

PostPosted: Tue Apr 03, 2012 3:29 pm    Post subject: Reply with quote

raynebc wrote:
I'm glad you're willing to guide the blind, but I still can't find what 0xD8C7024A is supposed to be a checksum for. It's not the checksum for the text on, On, \L=on, "\L=on" or even the entire line of text e1b093e2 "\L=on". That checksum is also not given anywhere else within the PAK file so it must be generated against text instead of being stored as a checksum someplace.

I see this line:
Code:
d8c7024a "\LOn"

_________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs



Back to top
View user's profile Wiki User Page Send private message Send e-mail Visit poster's website XBL Gamertag: codemann8 PSN Name: codemann8
codemann8  





Joined: 10 Apr 2007
Posts: 1045
Location: Oshkosh, WI

PostPosted: Tue Apr 03, 2012 6:02 pm    Post subject: Reply with quote

skylerinman wrote:
codemann8 wrote:
I don't know what you mean, this thread is purely an informational thread, not looking for anyone to test anything


I'm aware, but there's always the possibility.

Fair enough
_________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs



Back to top
View user's profile Wiki User Page Send private message Send e-mail Visit poster's website XBL Gamertag: codemann8 PSN Name: codemann8
raynebc  





Joined: 16 Jun 2008
Posts: 983

PostPosted: Tue Apr 03, 2012 6:06 pm    Post subject: Reply with quote

That makes me feel silly, I was looking for the entry for "on" and not "On". Is there a file header for the QS file that I should be looking for, or something like a magic number like with the QB files? Thank you again for all of your help!
Back to top
View user's profile Send private message
codemann8  





Joined: 10 Apr 2007
Posts: 1045
Location: Oshkosh, WI

PostPosted: Tue Apr 03, 2012 7:54 pm    Post subject: Reply with quote

raynebc wrote:
That makes me feel silly, I was looking for the entry for "on" and not "On". Is there a file header for the QS file that I should be looking for, or something like a magic number like with the QB files? Thank you again for all of your help!
Again, its a text file, not a binary file. That means no headers. It's not meant to have any structured format except that each line is it's own record. Best way to handle the file is to read in the whole file into a buffer and explode() on the 0x0D0A character sequence to get all entries, and then each record is really a hash (don't bother calculating the checksum on the string, just store what's given to you)...read in 8 characters for the key (you could convert the text to a unsigned int if you want, depending on what language ur writing ur stuff in, theres prob a library that converts that for you), skip the space, and the rest of the line is the value. Store that information in a hash object array and perform a lookup against it later
_________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs



Back to top
View user's profile Wiki User Page Send private message Send e-mail Visit poster's website XBL Gamertag: codemann8 PSN Name: codemann8
raynebc  





Joined: 16 Jun 2008
Posts: 983

PostPosted: Tue Apr 03, 2012 8:55 pm    Post subject: Reply with quote

I didn't want to take it for granted that it doesn't have a header of some kind. Headers for text files are less common, but they're not unheard of (even FoF's song.ini files have a header of "[song]"). I'll probably have to do something a bit more specific than searching for instances of 0x0D0A though, that hex string occurs twice in the Hotel California file before the QS data is reached, and even then it occurs after the first lyric entry. I think I could rely on searching for the first occurrence of 0x20225C4C ( "\L) though, and then rewind to the first of the preceding alphanumerical characters.

From there, I'll probably just build a list of strings and their hash values. This will account for entries being used more than once, just in case the files happen to do that. At least I've got a plan for moving ahead with this, I thank you again for your guidance!
Back to top
View user's profile Send private message
codemann8  





Joined: 10 Apr 2007
Posts: 1045
Location: Oshkosh, WI

PostPosted: Tue Apr 03, 2012 9:06 pm    Post subject: Reply with quote

I'm curious, what language are you coding in to read these files?

And splitting by 0x0D0A would most definitely be legit, I see in this specific example there are a couple of null entries at the end, but handling that case is super easy...there's only one 0x0D0A between each record in every case
_________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs



Back to top
View user's profile Wiki User Page Send private message Send e-mail Visit poster's website XBL Gamertag: codemann8 PSN Name: codemann8
raynebc  





Joined: 16 Jun 2008
Posts: 983

PostPosted: Tue Apr 03, 2012 9:47 pm    Post subject: Reply with quote

I mean to say that the PAK file as a whole has 0x0D0A instances before the embedded QS file is reached. I'm programming in old fashioned C, buffering the entire PAK file into memory and then parsing it from there instead of extracting the individual files (QB, QS, etc). It's not an extremely elegant solution, but the PAK file is otherwise pretty self contained and has just about everything I need to decode the contents (song name, instrument notes, vocals, etc).
Back to top
View user's profile Send private message
codemann8  





Joined: 10 Apr 2007
Posts: 1045
Location: Oshkosh, WI

PostPosted: Tue Apr 03, 2012 10:26 pm    Post subject: Reply with quote

raynebc wrote:
buffering the entire PAK file into memory and then parsing it from there instead of extracting the individual files (QB, QS, etc).

Eww, yuck, that means all the file offsets inside the QB file will not be correct (or you'll have to add that offset to the offset of the start of the QB file)...at the very least, you should write a wrapper for extracting QB and QS files out of the PAK separately in memory, that way you can still just input the one PAK file but then reference the sub files separately. You're right tho, the PAK is more convenient, that everything is contained.

If you want to parse out the PAK correctly, use this spec: http://slowhero.moto-coda.org/tech/pak.file.spec.txt, its actually a simple file structure
_________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs



Back to top
View user's profile Wiki User Page Send private message Send e-mail Visit poster's website XBL Gamertag: codemann8 PSN Name: codemann8
raynebc  





Joined: 16 Jun 2008
Posts: 983

PostPosted: Tue Apr 03, 2012 11:21 pm    Post subject: Reply with quote

It's not that bad, I just search for the QB magic number and use the offset of the QB file header, just as you mention. I don't really want to go to all the effort of parsing everything formally, I just hunt and peck for what I'm interested in (ie. instrument tracks, star power sections, etc).
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    ScoreHero Forum Index -> Software All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6, 7  Next
Page 5 of 7

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




NEW SERVER
Copyright © 2006-2019 ScoreHero, LLC
Terms of Use | Privacy Policy


Powered by phpBB