Star Power Optimization Script

[youhas]

As mentioned a while back, I was tinkering with a star power optimization script. Plink in all the notes and star power sequences; let Scripting Technology(tm) grind away at it for a bit; out comes a neat little brick of "when-to-activate" messages. Figured I'd at least clue everyone in on the results of that effort.

Call a "tick" the smallest unit of granularity you'd like to consider between potential SP activations. One tick every quarter beat works well enough for most songs. I've numbered the ticks for a test song - "Sail Your Ship By" on Medium - like so:
http://www.antwon.com/gh/sail-your-ship-by-2.png

I then painstakingly convert that series of notes into a tick-by-tick iteration of the notes, holds, and star power sequences in the song, which looks like this:
http://www.antwon.com/gh/SYSB_Medium.txt

At that point, fire up the script, and in five or ten minutes I get the following:
http://www.antwon.com/gh/SYSB_Medium_Results.txt
...a series of "ticks to activate SP on and the ticks that SP will be gone by" events. Which is good enough for 2nd place on Medium, and probably falls short only because of "implementation error" (read as: I don't squeeze notes perfectly and don't always whammy the shorter holds). (Not that I know for certain, of course.)

The script has some shortcomings. My culling algorithm is a touch overly aggressive, so I occasionally have to nudge the activations a tick or two later to get in a squeeze that it ignores. (That or turn off culling, which makes the script run for hours instead of minutes.) Notes are seen as single points instead of as they are, where there are note-hitting windows, which means that while it handles single-squeezes OK, double-squeeze opportunities are ignored. It also assumes, incorrectly, that SP can be activated instantaneously upon getting a half-bar. So it's not perfect, not by any means... but it works well enough for my purposes.

Originally, I was going to do the whole "extract the notes from the .iso and convert them into a script-accessible format" song-and-dance... but I'm just never going to find the time. Between that, the light shortcomings listed above, and that pretty much every Expert song has already been combed over by our resident SP-optimizing team, I kind of let things die out in the "proof of concept" stage you see above.

That said: if anyone wants to deal with the whole extract-the-notes side of the problem and/or has a solution to shove my way, please do. (And hell, if you want to laboriously hand-code some note charts of your own, I'll be happy to run the script against them and see what comes out - just shoot those text files my way and I'll run 'em through.)
_________________

Amusing the world 140 characters at a time: http://www.twitter.com/youhas

[youhas]

As mentioned a while back, I was tinkering with a star power optimization script. Plink in all the notes and star power sequences; let Scripting Technology(tm) grind away at it for a bit; out comes a neat little brick of "when-to-activate" messages. Figured I'd at least clue everyone in on the results of that effort.

Call a "tick" the smallest unit of granularity you'd like to consider between potential SP activations. One tick every quarter beat works well enough for most songs. I've numbered the ticks for a test song - "Sail Your Ship By" on Medium - like so:
http://www.antwon.com/gh/sail-your-ship-by-2.png

I then painstakingly convert that series of notes into a tick-by-tick iteration of the notes, holds, and star power sequences in the song, which looks like this:
http://www.antwon.com/gh/SYSB_Medium.txt

At that point, fire up the script, and in five or ten minutes I get the following:
http://www.antwon.com/gh/SYSB_Medium_Results.txt
...a series of "ticks to activate SP on and the ticks that SP will be gone by" events. Which is good enough for 2nd place on Medium, and probably falls short only because of "implementation error" (read as: I don't squeeze notes perfectly and don't always whammy the shorter holds). (Not that I know for certain, of course.)

The script has some shortcomings. My culling algorithm is a touch overly aggressive, so I occasionally have to nudge the activations a tick or two later to get in a squeeze that it ignores. (That or turn off culling, which makes the script run for hours instead of minutes.) Notes are seen as single points instead of as they are, where there are note-hitting windows, which means that while it handles single-squeezes OK, double-squeeze opportunities are ignored. It also assumes, incorrectly, that SP can be activated instantaneously upon getting a half-bar. So it's not perfect, not by any means... but it works well enough for my purposes.

Originally, I was going to do the whole "extract the notes from the .iso and convert them into a script-accessible format" song-and-dance... but I'm just never going to find the time. Between that, the light shortcomings listed above, and that pretty much every Expert song has already been combed over by our resident SP-optimizing team, I kind of let things die out in the "proof of concept" stage you see above.

That said: if anyone wants to deal with the whole extract-the-notes side of the problem and/or has a solution to shove my way, please do. (And hell, if you want to laboriously hand-code some note charts of your own, I'll be happy to run the script against them and see what comes out - just shoot those text files my way and I'll run 'em through.)
_________________

Amusing the world 140 characters at a time: http://www.twitter.com/youhas

[thekovinc]

This is pretty awesome...

I should totally do something like this for GH II.... or you should :P
_________________

Be sure to check out my YouTube vids here!

[thekovinc]

This is pretty awesome...

I should totally do something like this for GH II.... or you should :P
_________________

Be sure to check out my YouTube vids here!

[youhas]

I'm sort of hoping that all of the scoring and star power accrual/depletion logic stays more-or-less the same between GH1 and GH2 - then I can just draw up similar charts for GH2 songs and plug 'em on in. 'Cuz hell, the script's already written, and that's the challenging part; manually generating data to run through it is a massively tedious but not terribly difficult affair.

What I really hope is that when some enterprising soul (like JCirri) is doing the data-mining for GH2 note charts, I can piggyback onto their efforts. Because any sort of "here are all the notes in a big messy text file, have fun" assistance means that I can do scripting for the note-data side of the divide, too. And if I've got scripting action for both the data-entry and data-processing sides of the equation... well, hell - I'll just start the damn thing running before I leave work for the weekend and on Monday morning, voila, optimized SP charts as far as the eye can see.
_________________

Amusing the world 140 characters at a time: http://www.twitter.com/youhas

[youhas]

I'm sort of hoping that all of the scoring and star power accrual/depletion logic stays more-or-less the same between GH1 and GH2 - then I can just draw up similar charts for GH2 songs and plug 'em on in. 'Cuz hell, the script's already written, and that's the challenging part; manually generating data to run through it is a massively tedious but not terribly difficult affair.

What I really hope is that when some enterprising soul (like JCirri) is doing the data-mining for GH2 note charts, I can piggyback onto their efforts. Because any sort of "here are all the notes in a big messy text file, have fun" assistance means that I can do scripting for the note-data side of the divide, too. And if I've got scripting action for both the data-entry and data-processing sides of the equation... well, hell - I'll just start the damn thing running before I leave work for the weekend and on Monday morning, voila, optimized SP charts as far as the eye can see.
_________________

Amusing the world 140 characters at a time: http://www.twitter.com/youhas

[Riz]

[Riz]

[JCirri]

Question: are you considering time signature in your script (in your sample file I didn't see it)? The duration of SP depends on time signature so if not it wouldn't be accurate.
_________________

[JCirri]

Question: are you considering time signature in your script (in your sample file I didn't see it)? The duration of SP depends on time signature so if not it wouldn't be accurate.
_________________

[Riz]

[Riz]

[youhas]

[nods] Yup, Riz has got it. (I should've added a link to the "I Love Rock and Roll" transcript, where you can see the measure changing with some regularity.)

I remember giving the whole "let me increase the granularity by an order of magnitude" a shot, just to see how badly that affected run times, but can't remember the results of that experiment. Might've been as bad as a straight order of magnitude slower, but don't hold me to that.

Either way, I'd be ecstatic with absolutely anything in a text file of the "events at timestamps" variety. (Not like my input format is cast in stone, after all.) Downscaling, reassembling, tweaking the input-massaging - I can handle anything of that variety. It's getting from "mash of bits in a midi file" to "timestamped events, ready for analyzing" that's the toughest nut to crack. (I know I can do it myself... but enduring a brick of trial-and-error to reinvent the wheel is offputting enough that I haven't bothered.)
_________________

Amusing the world 140 characters at a time: http://www.twitter.com/youhas

[youhas]

[nods] Yup, Riz has got it. (I should've added a link to the "I Love Rock and Roll" transcript, where you can see the measure changing with some regularity.)

I remember giving the whole "let me increase the granularity by an order of magnitude" a shot, just to see how badly that affected run times, but can't remember the results of that experiment. Might've been as bad as a straight order of magnitude slower, but don't hold me to that.

Either way, I'd be ecstatic with absolutely anything in a text file of the "events at timestamps" variety. (Not like my input format is cast in stone, after all.) Downscaling, reassembling, tweaking the input-massaging - I can handle anything of that variety. It's getting from "mash of bits in a midi file" to "timestamped events, ready for analyzing" that's the toughest nut to crack. (I know I can do it myself... but enduring a brick of trial-and-error to reinvent the wheel is offputting enough that I haven't bothered.)
_________________

Amusing the world 140 characters at a time: http://www.twitter.com/youhas

[JCirri]

Ah ok, I didn't notice the measure keyword.

I'm still working myself on an algorithm that will take into account squeezing (exact amounts based on the allowable timing window for the given difficulty and at the current tempo at that point in the song), and also the delay it takes to activate SP.

However, I can probably provide you some files in the exact format that you use if you want to test it out. Except for the GRYBO parts, my program only cares if it's a note or a chord, and doesn't care which actual frets they are. So I could just make every note displayed as G and every chord GR or something if that would work in your program, let me know.

I have some time this weekend so I'm gonna try to get my program functional, then we can compare our results.
_________________

[JCirri]

Ah ok, I didn't notice the measure keyword.

I'm still working myself on an algorithm that will take into account squeezing (exact amounts based on the allowable timing window for the given difficulty and at the current tempo at that point in the song), and also the delay it takes to activate SP.

However, I can probably provide you some files in the exact format that you use if you want to test it out. Except for the GRYBO parts, my program only cares if it's a note or a chord, and doesn't care which actual frets they are. So I could just make every note displayed as G and every chord GR or something if that would work in your program, let me know.

I have some time this weekend so I'm gonna try to get my program functional, then we can compare our results.
_________________

[Riz]

I'm also in the process right now of trying to hack something together. We'll see how it goes.

Riz.
_________________

[Riz]

I'm also in the process right now of trying to hack something together. We'll see how it goes.

Riz.
_________________

[youhas]

Yep, making every note a G and every chord a GR would work just fine. (I only made the distinction between frets to help with the hand-transcribing part of things.) But sure - if it's feasible for you to create some files of your choosing to feed into the script in the near future, that'd be awesome.

I knew that you were working along similar lines, taking a lot more of the scoring nuances into account. Here's to hoping that you can complete your optimization program soon enough - I'd love to compare results and see how closely they match up.
_________________

Amusing the world 140 characters at a time: http://www.twitter.com/youhas

[youhas]

Yep, making every note a G and every chord a GR would work just fine. (I only made the distinction between frets to help with the hand-transcribing part of things.) But sure - if it's feasible for you to create some files of your choosing to feed into the script in the near future, that'd be awesome.

I knew that you were working along similar lines, taking a lot more of the scoring nuances into account. Here's to hoping that you can complete your optimization program soon enough - I'd love to compare results and see how closely they match up.
_________________

Amusing the world 140 characters at a time: http://www.twitter.com/youhas

[Riz]

[reposting to make the topic shiny and new... too many changes, but didn't want to double post...]

OK, extensive status re-update here. It's done. It appears to work perfectly, as far as I can see, at least in comparison to youhas's SYSB/med test dataset.

The script actually caught a number of fairly neat things.

A couple at the beginning are just wrong notes, which doesn't really change anything important... they're chords, and your dataset has them as chords... mine just has them as different chords, so no biggie.

The first really intersting catch, however is at tick 1516, which your dataset shows as hold=8 star13, and mine shows as hold=10 star13. That one matters. 2 extra ticks (half a beat) of starpower are available there.

The next catch is at 1949.333 and 1950.666. These Y and R notes you have listed at 1950 and 1951. Little bit of sneaky triplet timing action there, and very difficult to see on the PNG, but you can see it. Same at 1997.333 and 1998.666 which you show at 1998 and 1999 respectively.

a buglet removed a paragraph that was here

Anyway... done. It outputs data in exactly your format. If you have them, I'd love to check another of your hand coded files against my output and against the PNG for double verification. Anything you got would be appreciated. If that all checks out, I will run the full dataset for all 196 files. edit to add: I have checked out my output against a few hard and expert PNGs, and in general, they look pretty good... however, I also just re-read Riff's rules for note handling, and I'm pretty positive that my code will not correctly handle either of his rule #5's... I'll work on that tomorrow, have a look at the charts for COTC and YGATC along with the raw data and see if I can figure out exactly what gets ignored and what the conditions are... it's a bit hard to tell from just his description, at least for me.)

Again, three cheers for CPAN and the MIDI module there. Made this a lot easier than it otherwise would have been. And also, thanks to youhas, who gave me some motivation to get up off my ass (well, figuratively) and code something fun for a change... haven't done that for a while...

Riz.

edit: updated status 0037 EDT
edit2: status re-updated 0129 EDT
edit3: fixed a bug that removed some of the differences between the datasets.
_________________

[Riz]

[reposting to make the topic shiny and new... too many changes, but didn't want to double post...]

OK, extensive status re-update here. It's done. It appears to work perfectly, as far as I can see, at least in comparison to youhas's SYSB/med test dataset.

The script actually caught a number of fairly neat things.

A couple at the beginning are just wrong notes, which doesn't really change anything important... they're chords, and your dataset has them as chords... mine just has them as different chords, so no biggie.

The first really intersting catch, however is at tick 1516, which your dataset shows as hold=8 star13, and mine shows as hold=10 star13. That one matters. 2 extra ticks (half a beat) of starpower are available there.

The next catch is at 1949.333 and 1950.666. These Y and R notes you have listed at 1950 and 1951. Little bit of sneaky triplet timing action there, and very difficult to see on the PNG, but you can see it. Same at 1997.333 and 1998.666 which you show at 1998 and 1999 respectively.

a buglet removed a paragraph that was here

Anyway... done. It outputs data in exactly your format. If you have them, I'd love to check another of your hand coded files against my output and against the PNG for double verification. Anything you got would be appreciated. If that all checks out, I will run the full dataset for all 196 files. edit to add: I have checked out my output against a few hard and expert PNGs, and in general, they look pretty good... however, I also just re-read Riff's rules for note handling, and I'm pretty positive that my code will not correctly handle either of his rule #5's... I'll work on that tomorrow, have a look at the charts for COTC and YGATC along with the raw data and see if I can figure out exactly what gets ignored and what the conditions are... it's a bit hard to tell from just his description, at least for me.)

Again, three cheers for CPAN and the MIDI module there. Made this a lot easier than it otherwise would have been. And also, thanks to youhas, who gave me some motivation to get up off my ass (well, figuratively) and code something fun for a change... haven't done that for a while...

Riz.

edit: updated status 0037 EDT
edit2: status re-updated 0129 EDT
edit3: fixed a bug that removed some of the differences between the datasets.
_________________

[b3nz0r]

Dude I understand the concept of this but have no idea what you guys are doing, but keep doing it, because this is awesome.
_________________

[b3nz0r]

Dude I understand the concept of this but have no idea what you guys are doing, but keep doing it, because this is awesome.
_________________

[Riz]

[Riz]

[youhas]

Hooray for Riz getting his geek on! Man, if I'd known this ever would've advanced beyond "tinkertoy novelty for me to net some top-10 medium scores", I might've actually, y'know, made a more concentrated effort to improve the algorithm's accuracy. No time like the present to start, though....

But wow - this is all turning out fairly awesome right here. Thanks for seeing how closely the hand-scribed data comes to your script-processed data. I know that there are artifacts in my data sets, both as pure errata (I botched some note or hold outright, d'oh) and intentional shortcuts (taking triplets and jamming them into the closest available half-beat markers, hoping it won't make a difference); it will be fantastic if those can disappear through the Magic of Technology(tm).

I do have something like half-a-dozen hand-coded charts on hand, though they're at work. (Naturally - where else would I work on my personal projects during "lunch hours"? ) They're yours for the analyzing, of course. If I get super-motivated, I'll race over there and get them up this weekend; worst case scenario, late Monday morning.

Either way, I'm pretty amped about this whole "multiple independent note-chewing systems" approach we're congealing into here. Makes it more likely that any given team will have a bug, of course... but also makes it waaay more likely that any oddities caused by the two systems' failure to mesh will be shaken out sooner than later. Sweet!
_________________

Amusing the world 140 characters at a time: http://www.twitter.com/youhas

[youhas]

Hooray for Riz getting his geek on! Man, if I'd known this ever would've advanced beyond "tinkertoy novelty for me to net some top-10 medium scores", I might've actually, y'know, made a more concentrated effort to improve the algorithm's accuracy. No time like the present to start, though....

But wow - this is all turning out fairly awesome right here. Thanks for seeing how closely the hand-scribed data comes to your script-processed data. I know that there are artifacts in my data sets, both as pure errata (I botched some note or hold outright, d'oh) and intentional shortcuts (taking triplets and jamming them into the closest available half-beat markers, hoping it won't make a difference); it will be fantastic if those can disappear through the Magic of Technology(tm).

I do have something like half-a-dozen hand-coded charts on hand, though they're at work. (Naturally - where else would I work on my personal projects during "lunch hours"? ) They're yours for the analyzing, of course. If I get super-motivated, I'll race over there and get them up this weekend; worst case scenario, late Monday morning.

Either way, I'm pretty amped about this whole "multiple independent note-chewing systems" approach we're congealing into here. Makes it more likely that any given team will have a bug, of course... but also makes it waaay more likely that any oddities caused by the two systems' failure to mesh will be shaken out sooner than later. Sweet!
_________________

Amusing the world 140 characters at a time: http://www.twitter.com/youhas

[b3nz0r]

I mean I know what you're doing, but I don't know how exactly it works. But I like it.

/gives this post the b3nz0r STAMP OF APPROVAL™

/stamps

Riz and youhas for president.
_________________

[b3nz0r]

I mean I know what you're doing, but I don't know how exactly it works. But I like it.

/gives this post the b3nz0r STAMP OF APPROVAL™

/stamps

Riz and youhas for president.
_________________