6 replies to this topic

[Sasha Al'Therin]

Since I've no idea where to report any issues with this mod, I'll do it here in the mega-mod section and mayhap someone knows where it should go and can re-direct the information.

helmla.itm

in the last and 4th ability the 1 effect present states that it is at index 15. This is wrong and will cause out of bound read errors for any subsequent mod that tries to read the file. It is actually at index 14.

this is a tp2 copy block (needs to be put inside an actual mod setup) that will fix the error in question. It is preferable to be ran immediately after all the NEJ stuff or it could be tacked onto the end of the last tp2 being used to patch NEJ Running it later may cause it to not work right as it is possible for other mods to modify the file in between without hitting the out of bound read error.

Spoiler

COPY_EXISTING ~helmla.itm~ ~override~
PATCH_IF (SOURCE_SIZE > 0x71) THEN BEGIN // protects against invalid files
  READ_LONG   0x64 "abil_off"
  READ_SHORT  0x68 "abil_num"
  READ_LONG   0x6a "fx_off"
  READ_SHORT  0x70 "fx_num"
  FOR (index = 0; index < abil_num; index = index + 1) BEGIN // loop through abilities
   READ_SHORT  ("%abil_off%" + 0x1e + ("%index%" * 0x38)) "abil_fx_num"
   READ_SHORT  ("%abil_off%" + 0x20 + ("%index%" * 0x38)) "abil_fx_idx"
   // patch only if the ability effect index = 15 & the ability is the last ability in the file
   PATCH_IF (%abil_fx_idx% = 15) AND (%index% = %abil_num% - 1) BEGIN
    WRITE_SHORT  ("%abil_off%" + 0x20 + ("%index%" * 0x38)) (%abil_fx_idx% - 1)
   END
  END
END
BUT_ONLY

[Lollorian]

WeiDU's so awesome

I used your code to check for any other items with potential index stuff like that and ended up modding the BWP's creature offset checker into this monstrosity (mad props to the G3 Debugging Suite too )

Spoiler

PATCH_IF (SOURCE_SIZE > 0x72) BEGIN
					READ_LONG  0x64 "abil_off"
					READ_SHORT 0x68 "abil_num"
					READ_LONG  0x6a "fx_off"
					READ_SHORT 0x70 "fx_num"
					SET "running_index" = "fx_num"
								
					FOR (index = 0; index < abil_num; index = index + 1) BEGIN
						READ_SHORT  ("%abil_off%" + 0x1e + ("%index%" * 0x38)) "abil_fx_num"
						READ_SHORT  ("%abil_off%" + 0x20 + ("%index%" * 0x38)) "abil_fx_idx"
						PATCH_IF (%running_index% != %abil_fx_idx%) BEGIN
					//		WRITE_SHORT  ("%abil_off%" + 0x20 + ("%index%" * 0x38)) (%running_index%)
							PATCH_PRINT "%SOURCE_FILE% error: Misindexed Effect #%abil_fx_idx% (possible correct index => #%running_index%)"
							SET "running_index" = "abil_fx_idx"
						END ELSE
						BEGIN
							SET "running_index" = "running_index" + "abil_fx_num"
						END
					END
				END

Ran this on an extracted NeJv691 setup and this is what it caught

Spoiler

[WeiDU.exe] WeiDU version 23100
[WeiDU.exe] Using scripting style "BG2"

Installing [Item Offset checker] [v1]
Copying and patching 1 file ...

Auril.ITM error: Misindexed Effect #12 (possible correct index => #11)

CELEBRA.ITM error: Misindexed Effect #19 (possible correct index => #18)

HELMLA.ITM error: Misindexed Effect #15 (possible correct index => #14)

J2HAXE.ITM error: Misindexed Effect #8 (possible correct index => #6)



SUCCESSFULLY INSTALLED	  Item Offset checker

Could you please check if the other 3 items that it warns about have the same problem?

Also... I'm evil so I ran this on the folder where I've extracted the mods I usually play with And lookey lookey...

Spoiler

[WeiDU.exe] WeiDU version 23104
This is a non-stable version. Unless you're sure about what you're doing, consid
er downgrading.
[WeiDU.exe] Using scripting style "BG2"

Installing [Item Offset checker] [v1]
Copying and patching 1 file ...

KiyoBow2.ITM error: Misindexed Effect #3 (possible correct index => #5)

KiyoBow2.ITM error: Misindexed Effect #3 (possible correct index => #5)

CELEBRA.ITM error: Misindexed Effect #20 (possible correct index => #19)

solasna5.itm error: Misindexed Effect #6 (possible correct index => #9)

CMBRQ05.ITM error: Misindexed Effect #0 (possible correct index => #1)

CMDAQ05.ITM error: Misindexed Effect #3 (possible correct index => #4)

CMHAQ02.ITM error: Misindexed Effect #5 (possible correct index => #3)

CMHAQ02.ITM error: Misindexed Effect #6 (possible correct index => #4)

CMHAQ02.ITM error: Misindexed Effect #3 (possible correct index => #5)

CMHAQ02.ITM error: Misindexed Effect #4 (possible correct index => #6)

CMS1H01.ITM error: Misindexed Effect #3 (possible correct index => #4)

CMS1H07.ITM error: Misindexed Effect #2 (possible correct index => #3)

CMS1H08.ITM error: Misindexed Effect #3 (possible correct index => #4)

CMS1H10.ITM error: Misindexed Effect #2 (possible correct index => #3)

CMS1H11.ITM error: Misindexed Effect #2 (possible correct index => #3)

CMSPQ02.ITM error: Misindexed Effect #3 (possible correct index => #13)

CMSTAF01.ITM error: Misindexed Effect #0 (possible correct index => #3)

CMSTQ02.ITM error: Misindexed Effect #9 (possible correct index => #7)

CMSTQ02.ITM error: Misindexed Effect #7 (possible correct index => #<img src='http://www.shsforums.net/public/style_emoticons/<#EMO_DIR#>/cool.png' class='bbc_emoticon' alt='8)' />

CMSTQ03.ITM error: Misindexed Effect #9 (possible correct index => #7)

CMSTQ03.ITM error: Misindexed Effect #7 (possible correct index => #<img src='http://www.shsforums.net/public/style_emoticons/<#EMO_DIR#>/cool.png' class='bbc_emoticon' alt='8)' />

CMSTQ04.ITM error: Misindexed Effect #9 (possible correct index => #7)

CMSTQ04.ITM error: Misindexed Effect #7 (possible correct index => #<img src='http://www.shsforums.net/public/style_emoticons/<#EMO_DIR#>/cool.png' class='bbc_emoticon' alt='8)' />

CMSTQ05.ITM error: Misindexed Effect #9 (possible correct index => #7)

CMSTQ05.ITM error: Misindexed Effect #7 (possible correct index => #<img src='http://www.shsforums.net/public/style_emoticons/<#EMO_DIR#>/cool.png' class='bbc_emoticon' alt='8)' />

CMSWQ03.ITM error: Misindexed Effect #5 (possible correct index => #10)

CMSWQ09.ITM error: Misindexed Effect #3 (possible correct index => #9)

CMSWQ11.ITM error: Misindexed Effect #3 (possible correct index => #4)

CMSWQ12.ITM error: Misindexed Effect #7 (possible correct index => #<img src='http://www.shsforums.net/public/style_emoticons/<#EMO_DIR#>/cool.png' class='bbc_emoticon' alt='8)' />

CMSWQ13.ITM error: Misindexed Effect #4 (possible correct index => #3)

CMSWQ14.ITM error: Misindexed Effect #3 (possible correct index => #4)

CMSWQ15.ITM error: Misindexed Effect #2 (possible correct index => #3)

CMSWQ17.ITM error: Misindexed Effect #3 (possible correct index => #<img src='http://www.shsforums.net/public/style_emoticons/<#EMO_DIR#>/cool.png' class='bbc_emoticon' alt='8)' />

MA2SW01.ITM error: Misindexed Effect #3 (possible correct index => #5)

MCMBOW01.ITM error: Misindexed Effect #2 (possible correct index => #3)

CMBRQ05.ITM error: Misindexed Effect #0 (possible correct index => #1)

CMDAQ01.ITM error: Misindexed Effect #3 (possible correct index => #4)

CMDAQ01.ITM error: Misindexed Effect #3 (possible correct index => #4)

CMDAQ05.ITM error: Misindexed Effect #3 (possible correct index => #4)

CMHAQ02.ITM error: Misindexed Effect #5 (possible correct index => #3)

CMHAQ02.ITM error: Misindexed Effect #6 (possible correct index => #4)

CMHAQ02.ITM error: Misindexed Effect #3 (possible correct index => #5)

CMHAQ02.ITM error: Misindexed Effect #4 (possible correct index => #6)

CMS1H01.ITM error: Misindexed Effect #3 (possible correct index => #4)

CMS1H07.ITM error: Misindexed Effect #2 (possible correct index => #3)

CMS1H08.ITM error: Misindexed Effect #3 (possible correct index => #4)

CMS1H10.ITM error: Misindexed Effect #2 (possible correct index => #3)

CMS1H11.ITM error: Misindexed Effect #2 (possible correct index => #3)

CMSPQ02.ITM error: Misindexed Effect #3 (possible correct index => #13)

CMSTAF01.ITM error: Misindexed Effect #0 (possible correct index => #3)

CMSTQ02.ITM error: Misindexed Effect #9 (possible correct index => #7)

CMSTQ02.ITM error: Misindexed Effect #7 (possible correct index => #<img src='http://www.shsforums.net/public/style_emoticons/<#EMO_DIR#>/cool.png' class='bbc_emoticon' alt='8)' />

CMSTQ03.ITM error: Misindexed Effect #9 (possible correct index => #7)

CMSTQ03.ITM error: Misindexed Effect #7 (possible correct index => #<img src='http://www.shsforums.net/public/style_emoticons/<#EMO_DIR#>/cool.png' class='bbc_emoticon' alt='8)' />

CMSTQ04.ITM error: Misindexed Effect #9 (possible correct index => #7)

CMSTQ04.ITM error: Misindexed Effect #7 (possible correct index => #<img src='http://www.shsforums.net/public/style_emoticons/<#EMO_DIR#>/cool.png' class='bbc_emoticon' alt='8)' />

CMSTQ05.ITM error: Misindexed Effect #9 (possible correct index => #7)

CMSTQ05.ITM error: Misindexed Effect #7 (possible correct index => #<img src='http://www.shsforums.net/public/style_emoticons/<#EMO_DIR#>/cool.png' class='bbc_emoticon' alt='8)' />

CMSWQ03.ITM error: Misindexed Effect #5 (possible correct index => #10)

CMSWQ09.ITM error: Misindexed Effect #3 (possible correct index => #9)

CMSWQ11.ITM error: Misindexed Effect #3 (possible correct index => #4)

CMSWQ12.ITM error: Misindexed Effect #7 (possible correct index => #<img src='http://www.shsforums.net/public/style_emoticons/<#EMO_DIR#>/cool.png' class='bbc_emoticon' alt='8)' />

CMSWQ13.ITM error: Misindexed Effect #0 (possible correct index => #4)

CMSWQ14.ITM error: Misindexed Effect #3 (possible correct index => #4)

CMSWQ15.ITM error: Misindexed Effect #2 (possible correct index => #3)

CMSWQ17.ITM error: Misindexed Effect #3 (possible correct index => #<img src='http://www.shsforums.net/public/style_emoticons/<#EMO_DIR#>/cool.png' class='bbc_emoticon' alt='8)' />

MA2SW01.ITM error: Misindexed Effect #3 (possible correct index => #5)

MCMBOW01.ITM error: Misindexed Effect #2 (possible correct index => #3)

AXENEJ01.ITM error: Misindexed Effect #1 (possible correct index => #4)

AXENEJ01.ITM error: Misindexed Effect #5 (possible correct index => #6)

CITTERN.ITM error: Misindexed Effect #1 (possible correct index => #0)

COFFEE.ITM error: Misindexed Effect #65534 (possible correct index => #0)

ELFWINE.ITM error: Misindexed Effect #65535 (possible correct index => #0)

wcwight.itm error: Misindexed Effect #2 (possible correct index => #1)

ZZA7CH.ITM error: Misindexed Effect #6 (possible correct index => #5)

DagSeb.ITM error: Misindexed Effect #10 (possible correct index => #9)

DeRing3.itm error: Misindexed Effect #42 (possible correct index => #41)

DeRing5.itm error: Misindexed Effect #70 (possible correct index => #68)

InvCloak.itm error: Misindexed Effect #6 (possible correct index => #5)

misc73n.itm error: Misindexed Effect #3 (possible correct index => #0)

TuPotnD1.itm error: Misindexed Effect #18 (possible correct index => #0)

TuSw1H02.itm error: Misindexed Effect #1 (possible correct index => #6)

UlFists.itm error: Misindexed Effect #0 (possible correct index => #2)

AXENEJ01.ITM error: Misindexed Effect #1 (possible correct index => #4)

AXENEJ01.ITM error: Misindexed Effect #5 (possible correct index => #6)

invcloak.itm error: Misindexed Effect #12 (possible correct index => #11)

SERPSCA.ITM error: Misindexed Effect #65535 (possible correct index => #0)


SUCCESSFULLY INSTALLED	  Item Offset checker

Edited by Lollorian, 25 April 2012 - 05:30 AM.

[Sasha Al'Therin]

Could you please check if the other 3 items that it warns about have the same problem?

I was just mass installing stuff with the BWS beta tool to see how it would handle. Ran into an issue that I couldn't (with my knowledge) repair and go on. So I had to wipe it and start over. currently don't have NEJ in a state to look at.

To confirm those items, all you need do is open NI and count all the effects up. In the case of helmla.itm there were 11 equipping effects and 4 abilities with 1 effect each. Since it's an index the count starts at 0. the abilities then should have been 11, 12, 13, 14 but they were 11, 12, 13, 15. apparently NI & DLTECP don't warn if the index is out of bounds, they just read the next unused effect. But weidu is more strict and has to use the value that's there.

I wonder what tool is used with these items to get them in this state. Seems that either when effects are removed the index doesn't get updated, or when effects are added the index is updated based on a start of 1 rather than 0. Which could be as that's been a long standing issue with modders, using 0 for #1 is a little odd....

[Lollorian]

... apparently NI & DLTECP don't warn if the index is out of bounds, they just read the next unused effect.

So that's why I wondered why the effect seemed to show up properly in NI

Are ability effects ALWAYS supposed to be ordered in increasing order corresponding to the order of the abilities themselves?
Are ability effects ALWAYS found after the global effects?
And finally... can some ability use a global effect as its ability effect?

[Sasha Al'Therin]

Are ability effects ALWAYS supposed to be ordered in increasing order corresponding to the order of the abilities themselves?
Are ability effects ALWAYS found after the global effects?
And finally... can some ability use a global effect as its ability effect?

in order of the questions:

so long as the effects are properly indexed they can be in differing order... i would imagine

99.9% of the time yes. theoretically they could be before so long as everything is properly indexed

I've not heard of one doing so, but it wouldn't surprise me if it were possible to do so.

Oh you were talking about Bioware stock?
yes
yes
no

[Sasha Al'Therin]

I ran similar code, but I like to use text to make sure I'm getting it right ^^ plus I put the output into an .txt file instead of having to extract from the debug... and to save on posting weidu.logs and tracking it down etc... automatic change-log ^^

code

Spoiler

BEGIN ~CodeTester #2: Check Ability Effect index Numbering~
<<<<<<<< inlined\blank.log
>>>>>>>>
COPY ~inlined\blank.log~ ~ab_BadEffectIndexes.txt~

ACTION_IF !(FILE_EXISTS ~weidu.exe~) THEN BEGIN
COPY ~setup-ab_CodeTester.exe~ ~weidu.exe~
END

COPY_EXISTING_REGEXP GLOB ~.*\.itm~ ~override~
PATCH_IF (SOURCE_SIZE > 0x72) BEGIN
  SPRINT file ~%SOURCE_FILE%~
  READ_LONG  0x64 "abil_off"
  READ_SHORT 0x68 "abil_num"
  READ_LONG  0x6a "fx_off"
  READ_SHORT 0x70 "fx_num"
  SET "running_index" = "fx_num"
  PATCH_IF (%fx_num% > 0) BEGIN
//   PATCH_PRINT ~%SOURCE_FILE%:  Scanning Ability Effects~
   FOR (index = 0; index < abil_num; index = index + 1) BEGIN
	READ_SHORT  ("%abil_off%" + 0x1e + ("%index%" * 0x38)) "abil_fx_num"
	READ_SHORT  ("%abil_off%" + 0x20 + ("%index%" * 0x38)) "abil_fx_idx"
	SET abil_count = %index% + 1
//	PATCH_PRINT ~Reading Ability #%index% which is %abil_count% of %abil_num% abilities%WNL%Ability Effect Index = %abil_fx_idx%%WNL%Ability Effect Number = %abil_fx_num%~
	FOR (idx2 = 0; idx2 < abil_fx_num; idx2 += 1) BEGIN
	 SET abil_fx_count = %idx2% + 1
//	 PATCH_PRINT ~Reading Ability Effect #%idx2% which is %abil_fx_count% of %abil_fx_num% ability effects~
	 SET abil_fx_loc = (%fx_off% + (%running_index% * 0x30))
	 SET this_fx_loc = (%idx2% * 0x30)
	 SET current_fx_idx = %abil_fx_idx% + %idx2%
	 PATCH_IF (%current_fx_idx% = %running_index%) BEGIN
	  SET running_index += 1
	 END ELSE
	 BEGIN
	  PATCH_PRINT ~%SOURCE_FILE%:%WNL%Ability #%index% has effect #%idx2% which reads out of bounds.%WNL%Listed effect index is (%abil_fx_idx% + %idx2%) = %current_fx_idx%.%WNL%Correct index should be %running_index%~
	  PATCH_PRINT ~halting ability effects read~
	  INNER_ACTION BEGIN
	   APPEND_OUTER ~ab_BadEffectIndexes.txt~ ~%SOURCE_FILE%:%WNL%Ability #%index% has effect #%idx2% which may read out of bounds.%WNL%Listed effect index is (%abil_fx_idx% + %idx2%) = %current_fx_idx%.%WNL%Correct index should be %running_index%~
	   AT_NOW  ~WeiDU.exe --log nul --change-log "%file%" --out ab_CodeTester\backup\change-log --append "ab_BadEffectIndexes.txt"~
	  END
	  //exit loop if 'bad'
	  SET idx2 = %abil_fx_num%
	 END
	END
   END
  END
END
BUT_ONLY
OUTER_SET showfile = 0
COPY ~ab_BadEffectIndexes.txt~ ~ab_BadEffectIndexes.txt~
SET showfile = 1
IF ~[a-z]~
BUT_ONLY

ACTION_IF showfile = 1 THEN BEGIN
AT_INTERACTIVE_EXIT ~VIEW ab_BadEffectIndexes.txt~
END

some sample results:
unmodified BG2 ToB
-- no results --

from NEJ2 only

Spoiler

HELMLA.ITM:
Ability #3 has effect #0 which may read out of bounds.
Listed effect index is (15 + 0) = 15.
Correct index should be 14

Mods affecting HELMLA.ITM:
00000: /* from game biffs */ ~SETUP-NEJ2V694.TP2~ 0 0 // Never Ending Journey Second Edition v6.94 (Patch)

note the other files you have listed are NOT in my game. must have been changed at some point between versions or I didn't include a particular component. I installed whatever the BWS beta tool said I needed.

from a 'Decent' install of mods couple big BG1 mods and smaller quest stuff

Spoiler

BOOT09.ITM:
Ability #0 has effect #0 which may read out of bounds.
Listed effect index is (0 + 0) = 0.
Correct index should be 1

Mods affecting BOOT09.ITM:
00000: /* from game biffs */ ~SETUP-UB.TP2~ 0 12 // Item Restorations: v19 BWP Fix
00001: ~AURORA/SETUP-AURORA.TP2~ 0 0 // Aurora's Shoes and Boots: v5

my original manual build of the 'decent' install above -- didn't use any BWP related stuff

Spoiler

BOOT09.ITM:
Ability #0 has effect #0 which may read out of bounds.
Listed effect index is (0 + 0) = 0.
Correct index should be 1

Mods affecting BOOT09.ITM:
00000: /* from game biffs */ ~SETUP-UB.TP2~ 0 12 // Item Restorations: v19
00001: ~AURORA/SETUP-AURORA.TP2~ 0 0 // Aurora's Shoes and Boots: v5

from my big ass everything the BWS beta is willing to throw at you.
-- nothing --
(it includes the NEJv264 as above, but I'd pre-patched it prior to install)


Try it on yours and see what you get. I don't know what mods you have, but I'm surprised that I didn't get many more hits since you got a ton.

I reported the boot09.itm to Aurora. I know it was that mod cause of the change log & when you look at their code you can see it updates the equipping effects but not the ability effects.

Edited by Sasha Al'Therin, 27 April 2012 - 12:27 AM.