View previous topic :: View next topic |
Author |
Message |
raynebc
Joined: 16 Jun 2008 Posts: 992
|
Posted: Mon Apr 02, 2012 7:20 pm Post subject: |
|
|
I did find that eventually there eventually appeared to be individual lyric entries in the PAK file, but I didn't recognize the format they're in. For example:
Code: | 0D 0A 65 39 65 62 36 31 66 36 20 22 5C 4C 64 61 72 6B 22 |
Those aren't all printable characters, but here's what that generally looks like in ASCII
Each lyric seems to end with a quote mark character, but that's the only trend I see here. These individual lyrics don't appear to be in order, and I don't see a timestamp like what we'd expect from GH5 files. Do you have any suggestions where I go from here? Is there a file spec for the QS format someplace? |
|
Back to top |
|
|
codemann8
Joined: 10 Apr 2007 Posts: 1048 Location: Oshkosh, WI
|
Posted: Mon Apr 02, 2012 7:43 pm Post subject: |
|
|
raynebc wrote: | I did find that eventually there eventually appeared to be individual lyric entries in the PAK file, but I didn't recognize the format they're in. For example:
Code: | 0D 0A 65 39 65 62 36 31 66 36 20 22 5C 4C 64 61 72 6B 22 |
Those aren't all printable characters, but here's what that generally looks like in ASCII
Each lyric seems to end with a quote mark character, but that's the only trend I see here. These individual lyrics don't appear to be in order, and I don't see a timestamp like what we'd expect from GH5 files. Do you have any suggestions where I go from here? Is there a file spec for the QS format someplace? | yep, qs files are text files, not binary files...the e9eb61f6 is the check sum and 'dark' is the lyric...the \L is on all the lyrics for some reason, just ignore it
As far as timestamps, looks like the 'On' lyric starts at 0xD8A4 ms, this is in the QB file _________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs
|
|
Back to top |
|
|
raynebc
Joined: 16 Jun 2008 Posts: 992
|
Posted: Mon Apr 02, 2012 9:02 pm Post subject: |
|
|
Doing a search for the first instance of the timestamp (0x0000D8A4) that occurs after the "hotelcalifornia_lyrics" header, I run into 40 byte entries for what appears to be each of the vocal notes. Here's the first one:
Code: | 00 00 D8 A4 00 02 49 14 00 01 1C 00 C4 74 58 38 D8 C7 02 4A 00 00 00 00 00 00 01 00 00 02 49 2C 00 01 01 00 90 6B 67 BA |
Here's the lyric entry for "on":
Code: | 0D 0A 65 31 62 30 39 33 65 32 20 22 5C 4C 3D 6F 6E 22 |
which looks like this in ASCII:
Each of the lyric entries does appear to be prefixed with 0x0D0A, but 0xe1b093e2 does not occur within the 40 byte entry. It does exist elsewhere in the PAK file though, but I don't see the timestamp nearby. I am definitely missing some of the pieces to this puzzle. |
|
Back to top |
|
|
codemann8
Joined: 10 Apr 2007 Posts: 1048 Location: Oshkosh, WI
|
Posted: Mon Apr 02, 2012 9:23 pm Post subject: |
|
|
raynebc wrote: | Doing a search for the first instance of the timestamp (0x0000D8A4) that occurs after the "hotelcalifornia_lyrics" header, I run into 40 byte entries for what appears to be each of the vocal notes. Here's the first one:
Code: | 00 00 D8 A4 00 02 49 14 00 01 1C 00 C4 74 58 38 D8 C7 02 4A 00 00 00 00 00 00 01 00 00 02 49 2C 00 01 01 00 90 6B 67 BA |
Here's the lyric entry for "on":
Code: | 0D 0A 65 31 62 30 39 33 65 32 20 22 5C 4C 3D 6F 6E 22 |
which looks like this in ASCII:
Each of the lyric entries does appear to be prefixed with 0x0D0A, but 0xe1b093e2 does not occur within the 40 byte entry. It does exist elsewhere in the PAK file though, but I don't see the timestamp nearby. I am definitely missing some of the pieces to this puzzle. | It's a text file, not a binary file, 0x0D0A represents the newline and carriage return 0x65 is 'e', 0x31 is '1', etc
And the first record in the lyrics section is (at 0x248FC):
Code: | 00 00 01 00
00 02 49 04 --offset to data (next byte)
00 01 01 00
90 6B 67 BA --some checksum (looks same for every lyric)
00 00 D8 A4 --start of lyric (ms)
00 02 49 14 --offset to the next data (next byte)
00 01 1C 00
C4 74 58 38 --some checksum (looks same for every lyric)
D8 C7 02 4A --checksum of lyric text (qs file)
00 00 00 00 |
_________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs
|
|
Back to top |
|
|
raynebc
Joined: 16 Jun 2008 Posts: 992
|
Posted: Mon Apr 02, 2012 11:14 pm Post subject: |
|
|
I'm glad you're willing to guide the blind, but I still can't find what 0xD8C7024A is supposed to be a checksum for. It's not the checksum for the text on, On, \L=on, "\L=on" or even the entire line of text e1b093e2 "\L=on". That checksum is also not given anywhere else within the PAK file so it must be generated against text instead of being stored as a checksum someplace. |
|
Back to top |
|
|
skylerinman
Joined: 03 Oct 2007 Posts: 329 Location: Gainesville, FL
|
Posted: Tue Apr 03, 2012 2:46 pm Post subject: |
|
|
codemann8 wrote: | I don't know what you mean, this thread is purely an informational thread, not looking for anyone to test anything |
I'm aware, but there's always the possibility. _________________
PS2 setup: Fat PS2 with Free MCboot v1.8, with a 120gb HD.
myvidsomg
Desertman123 is a God, he knows his shit. dclapp wrote: | Fork1 wrote: | I quit. This team is Deadhead6391s now. | you quit? so no more hacking YT accounts then? |
|
|
Back to top |
|
|
codemann8
Joined: 10 Apr 2007 Posts: 1048 Location: Oshkosh, WI
|
Posted: Tue Apr 03, 2012 3:29 pm Post subject: |
|
|
raynebc wrote: | I'm glad you're willing to guide the blind, but I still can't find what 0xD8C7024A is supposed to be a checksum for. It's not the checksum for the text on, On, \L=on, "\L=on" or even the entire line of text e1b093e2 "\L=on". That checksum is also not given anywhere else within the PAK file so it must be generated against text instead of being stored as a checksum someplace. |
I see this line: _________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs
|
|
Back to top |
|
|
codemann8
Joined: 10 Apr 2007 Posts: 1048 Location: Oshkosh, WI
|
Posted: Tue Apr 03, 2012 6:02 pm Post subject: |
|
|
skylerinman wrote: | codemann8 wrote: | I don't know what you mean, this thread is purely an informational thread, not looking for anyone to test anything |
I'm aware, but there's always the possibility. |
Fair enough _________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs
|
|
Back to top |
|
|
raynebc
Joined: 16 Jun 2008 Posts: 992
|
Posted: Tue Apr 03, 2012 6:06 pm Post subject: |
|
|
That makes me feel silly, I was looking for the entry for "on" and not "On". Is there a file header for the QS file that I should be looking for, or something like a magic number like with the QB files? Thank you again for all of your help! |
|
Back to top |
|
|
codemann8
Joined: 10 Apr 2007 Posts: 1048 Location: Oshkosh, WI
|
Posted: Tue Apr 03, 2012 7:54 pm Post subject: |
|
|
raynebc wrote: | That makes me feel silly, I was looking for the entry for "on" and not "On". Is there a file header for the QS file that I should be looking for, or something like a magic number like with the QB files? Thank you again for all of your help! | Again, its a text file, not a binary file. That means no headers. It's not meant to have any structured format except that each line is it's own record. Best way to handle the file is to read in the whole file into a buffer and explode() on the 0x0D0A character sequence to get all entries, and then each record is really a hash (don't bother calculating the checksum on the string, just store what's given to you)...read in 8 characters for the key (you could convert the text to a unsigned int if you want, depending on what language ur writing ur stuff in, theres prob a library that converts that for you), skip the space, and the rest of the line is the value. Store that information in a hash object array and perform a lookup against it later _________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs
|
|
Back to top |
|
|
raynebc
Joined: 16 Jun 2008 Posts: 992
|
Posted: Tue Apr 03, 2012 8:55 pm Post subject: |
|
|
I didn't want to take it for granted that it doesn't have a header of some kind. Headers for text files are less common, but they're not unheard of (even FoF's song.ini files have a header of "[song]"). I'll probably have to do something a bit more specific than searching for instances of 0x0D0A though, that hex string occurs twice in the Hotel California file before the QS data is reached, and even then it occurs after the first lyric entry. I think I could rely on searching for the first occurrence of 0x20225C4C ( "\L) though, and then rewind to the first of the preceding alphanumerical characters.
From there, I'll probably just build a list of strings and their hash values. This will account for entries being used more than once, just in case the files happen to do that. At least I've got a plan for moving ahead with this, I thank you again for your guidance! |
|
Back to top |
|
|
codemann8
Joined: 10 Apr 2007 Posts: 1048 Location: Oshkosh, WI
|
Posted: Tue Apr 03, 2012 9:06 pm Post subject: |
|
|
I'm curious, what language are you coding in to read these files?
And splitting by 0x0D0A would most definitely be legit, I see in this specific example there are a couple of null entries at the end, but handling that case is super easy...there's only one 0x0D0A between each record in every case _________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs
|
|
Back to top |
|
|
raynebc
Joined: 16 Jun 2008 Posts: 992
|
Posted: Tue Apr 03, 2012 9:47 pm Post subject: |
|
|
I mean to say that the PAK file as a whole has 0x0D0A instances before the embedded QS file is reached. I'm programming in old fashioned C, buffering the entire PAK file into memory and then parsing it from there instead of extracting the individual files (QB, QS, etc). It's not an extremely elegant solution, but the PAK file is otherwise pretty self contained and has just about everything I need to decode the contents (song name, instrument notes, vocals, etc). |
|
Back to top |
|
|
codemann8
Joined: 10 Apr 2007 Posts: 1048 Location: Oshkosh, WI
|
Posted: Tue Apr 03, 2012 10:26 pm Post subject: |
|
|
raynebc wrote: | buffering the entire PAK file into memory and then parsing it from there instead of extracting the individual files (QB, QS, etc). |
Eww, yuck, that means all the file offsets inside the QB file will not be correct (or you'll have to add that offset to the offset of the start of the QB file)...at the very least, you should write a wrapper for extracting QB and QS files out of the PAK separately in memory, that way you can still just input the one PAK file but then reference the sub files separately. You're right tho, the PAK is more convenient, that everything is contained.
If you want to parse out the PAK correctly, use this spec: http://slowhero.moto-coda.org/tech/pak.file.spec.txt, its actually a simple file structure _________________
Frets and Fireworks GH:A Expert Tourney Winner
AAA-2 Season 5 Overall Points Champ & Playoff Runner-Up!!!
My FCs
|
|
Back to top |
|
|
raynebc
Joined: 16 Jun 2008 Posts: 992
|
Posted: Tue Apr 03, 2012 11:21 pm Post subject: |
|
|
It's not that bad, I just search for the QB magic number and use the offset of the QB file header, just as you mention. I don't really want to go to all the effort of parsing everything formally, I just hunt and peck for what I'm interested in (ie. instrument tracks, star power sections, etc). |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Copyright © 2006-2024 ScoreHero, LLC
|
Powered by phpBB
|