Jump to content


Photo

Problems reading BAM files


  • Please log in to reply
8 replies to this topic

#1 MasterScrat

MasterScrat
  • Member
  • 13 posts

Posted 30 March 2015 - 02:17 PM

Hello everyone,
 
I am trying to read BAM files but I'm having some issues.
 
I parse the BAM header according to the IESDP doc, and get generally good results:
 
[signature] => BAM     [version] => V1      [nbFrames] => 135    [nbCycles] => 9    [transparentIndex] => 0    [offsetFrameEntries] => 24    [offsetPalette] => 1680    [offsetFrameLut] => 2704
 
That's for DBAM1A1.bam. This one works fine!
 
But now some other BAM files, like DBAM1G1.bam, are arranged in a different order:
 
[signature] => BAM     [version] => V1      [nbFrames] => 846    [nbCycles] => 99    [transparentIndex] => 0    [offsetFrameEntries] => 45728    [offsetPalette] => 24    [offsetFrameLut] => 1048
 
Here I have the palette, the frame lookup table then the frame entries (and, I guess, the cycle entries). That's problematic because I rely on the cycle entries to know how large the frame lookup table is! As per the IESDP:
 
To find the number of entries in this lookup table, find the largest value of start+count in the cycle entries table.
 
So I am stuck, I tried computing the lookup table size from (offsetFrameEntries - offsetFrameLut) but this doesn't work. Actually, parsing frames from the offsetFrameLut position doesn't even work!
 
Any idea? Am I missing something?

Edit: fixed file names

Edited by MasterScrat, 30 March 2015 - 11:11 PM.


#2 The Imp

The Imp

    Not good, see EVIL is better. You'll LIVE.

  • Member
  • 5150 posts

Posted 30 March 2015 - 03:02 PM

Erhm, in which game is this for ? Cause I am not aware that the infinity engines games actually holds more than 8 character/letter files ! +the obvious extension. (DKHALID1A1.bam has 10 characters + .bam extension)

Now if the .bam's are converted into another format and/or back with bamworkshop and bamworkshop2, thus creating isolated single frames, now then those can hold more than 8 characters, but they are not .bams anymore, but .png's or whatever.


Edited by The Imp, 30 March 2015 - 03:08 PM.

Yep, Jarno Mikkola. my Mega Mod FAQ. Use of the BWS, and how to use it(scroll down that post a bit). 
OK, desert dweller, welcome to the sanity, you are free to search for the limit, it's out there, we drew it in the sand. Ouh, actually it was still snow then.. but anyways.


#3 MasterScrat

MasterScrat
  • Member
  • 13 posts

Posted 30 March 2015 - 11:09 PM

Oops your are right, I made a mistake copy-pasting the names. I am using original, unedited BG2 files.

I meant: DBAM1A1.bam and DBAM1G1.bam. Which are used from DKHALID01.cre, if I remember well.

#4 Sam.

Sam.
  • Administrator
  • 1294 posts

Posted 31 March 2015 - 09:51 AM

Oops your are right, I made a mistake copy-pasting the names. I am using original, unedited BG2 files.

I meant: DBAM1A1.bam and DBAM1G1.bam. Which are used from DKHALID01.cre, if I remember well.

I don't see any of those three file names in my game (granted it is modded), and "DKHALID01.cre" is still bigger than the allowed 8.3 filename.  Please make sure you have the filenames correct, or even better upload them here for inspection.

 

I'm not sure what language you are working in, but you might find it useful to know how Beamdog loads BAM files.  Note that "bammer" is Beamdog's internal BAM editing tool.


"Ok, I've just about had my FILL of riddle asking, quest assigning, insult throwing, pun hurling, hostage taking, iron mongering, smart-arsed fools, freaks, and felons that continually test my will, mettle, strength, intelligence, and most of all, patience! If you've got a straight answer ANYWHERE in that bent little head of yours, I want to hear it pretty damn quick or I'm going to take a large blunt object roughly the size of Elminster AND his hat, and stuff it lengthwise into a crevice of your being so seldom seen that even the denizens of the nine hells themselves wouldn't touch it with a twenty-foot rusty halberd! Have I MADE myself perfectly CLEAR?!"

--<CHARNAME> to Portalbendarwinden

--------------------

post-10485-0-15080600-1348188745.jpg
___________Old pen and paper modules of the 70s and 80s.___________

CA Forums CA Homepage


#5 MasterScrat

MasterScrat
  • Member
  • 13 posts

Posted 31 March 2015 - 01:39 PM

I'm not sure what language you are working in, but you might find it useful to know how Beamdog loads BAM files.  Note that "bammer" is Beamdog's internal BAM editing tool.

 

When the hell did that come out. I spent so much time writing my own PHP converters 5 years ago (yes PHP, I was young and stupid at the time). This should solve a lot of my problems, thanks a lot for pointing it out to me! :D

 

Does anyone know why there are utilities to convert to PNG and JSON formats? Does anyone have a link to some kind of announcement?!

 

In any case this me very happy :D 



#6 Sam.

Sam.
  • Administrator
  • 1294 posts

Posted 31 March 2015 - 08:11 PM

Does anyone have a link to some kind of announcement?!

There hasn't been one.  I'll PM you some stuff, though.  It is important to note that bammer is NOT supported in any way by Beamdog or Scott Brooks.  What you see is what you get.


"Ok, I've just about had my FILL of riddle asking, quest assigning, insult throwing, pun hurling, hostage taking, iron mongering, smart-arsed fools, freaks, and felons that continually test my will, mettle, strength, intelligence, and most of all, patience! If you've got a straight answer ANYWHERE in that bent little head of yours, I want to hear it pretty damn quick or I'm going to take a large blunt object roughly the size of Elminster AND his hat, and stuff it lengthwise into a crevice of your being so seldom seen that even the denizens of the nine hells themselves wouldn't touch it with a twenty-foot rusty halberd! Have I MADE myself perfectly CLEAR?!"

--<CHARNAME> to Portalbendarwinden

--------------------

post-10485-0-15080600-1348188745.jpg
___________Old pen and paper modules of the 70s and 80s.___________

CA Forums CA Homepage


#7 fearless

fearless
  • Member
  • 79 posts

Donator

Posted 01 April 2015 - 01:13 PM

The FrameDataOffset can be obtained from the first frame entry, FrameEntry0, specifically offset 8 bytes from the start of the entry:

 

    bits 30-0: Offset to frame data
    bit 31: 0=Compressed (RLE), 1=Uncompressed

 

Once you have this value (FrameDataOffset) you can subtract FrameLookupOffset from it to obtain the size of the frame lookup table.

 

Looking at DEMB1A1.bam from MISCAnim.bif you can see the following:

- The framelookupoffset is 2704 (0x00000A90)

- The framedataoffset for frameentry0 is 2956 (0x00000B8C)

- Which gives us 252 bytes for our frame lookup table (2956 - 2704)

 

(However, each cycle can have a variable length of frames in it and the framelookuptable is a word sized array (2 bytes long) for each frame pointer)

 

 

Reading FrameLookupTable Frame Sequence:

 

To get the sequences of frames that play in a cycle:

- Read each cycle entry and get the cycleindex and cyclecount.

- Take the cycleindex and multiply it by 2 to get the offset in bytes to read from start of the framelookuptable and add the framelookupoffset value. This gives us our start of the cycle sequence.

- Read cyclecount x 2 amount of bytes to get the full sequence size for this particular cycle.

 

 

Example: (cycle0)

 

CycleEntries start at offset 1644 (0x0000066C) which is also the start of CycleEntry0

- CycleEntry0's data is 0E00 0000

- Count of frame indices in this cycle is (0x000E) 14

- Index into frame lookup table of first frame index in this cycle is (0x0000) 0

 

So framelookupoffset + (0 x 2) = 2704 + (0) which is the start sequence of cycle0

we read count of frames x 2 (14 x 2) = 28 bytes and we get this data:

 

0000 0100 0200 0300 0400 0500 0600 0700 0800 0900 0A00 0B00 0C00 0D00

 

Remember each entry of this sequence is a word length or 2 bytes long (ive put a space between each word value)

the values in that sequence directly refer to which frame (frame entry->framedataoffset->convert to bitmap) to show: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 - so we have 14 frames in this cycle, starting at frame0 and ending at frame13

 

 

Example: (cycle1)

- Cycle1's data is 0E00 0E00

- Count of frame indices in this cycle is (0x000E) 14

- Index into frame lookup table of first frame index in this cycle is (0x000E) 14

 

So framelookupoffset + (14 x 2) =  2704 + (28) = 2732 which is the start sequence of cycle1

we read count of frames x 2 (14 x 2) = 28 bytes and we get this data:

 

0F00 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 1A00 1B00 1C00

 

Again we have 14 frames to show, starting at frame 15 and ending at frame 28. As i mentioned above each cycle can have different amount of frames in its lookup sequence.

 

 

DEMB1G1.bam has a lot more frames, more cycles and more cycle sequences with different lengths, as can be seen in this wip screenshot from a program/utility i am writing (total column indicates no of frames in cycle animation sequence)

 

winbam-demb1g1.png

 

One oddity i noticed when looking at the data for all this, was that some frames dont appear in any cycle animation sequence. There are more frames than are actually used. in the above examples frame 14 is not used in cycle0 or cycle1, same with other bam files i have looked at - and bamworkshop also indicates this - shows all frames available, but for the defined cycles, some frames are not included. Strange? or am i missing something?

 


fearless

 

CM690 II Case, Corsair HX1000 PSU, Asus Maximus V Gene, Intel i5-3750K @ 4.2Ghz, Corsair 8GB DDR3, Asus DirectCU GTX-670, Samsung 256GB SSD, WD Caviar Black 2TB HDD x2

 

https://github.com/mrfearless


#8 Sam.

Sam.
  • Administrator
  • 1294 posts

Posted 02 April 2015 - 06:16 AM

The FrameDataOffset can be obtained from the first frame entry, FrameEntry0, specifically offset 8 bytes from the start of the entry:

    bits 30-0: Offset to frame data
    bit 31: 0=Compressed (RLE), 1=Uncompressed
Once you have this value (FrameDataOffset) you can subtract FrameLookupOffset from it to obtain the size of the frame lookup table.

This assumes that the Frame Data for the first Frame Entry is ordered first in the BAM. While this may appear to be a logical thing to assume, I don't think it is necessarily guaranteed. It would be safer to loop through all the Frame Entries and take the lowest value of FrameDataOffset to be the actual start of the Frame Data section. Note that a Frame Entry with Width and/or Height of zero (0) need not necessarily have a reasonable FrameDataOffset (as it would have no real frame data) and so should be excluded from consideration of the lowest value of FrameDataOffset.

 

One oddity i noticed when looking at the data for all this, was that some frames dont appear in any cycle animation sequence. There are more frames than are actually used. in the above examples frame 14 is not used in cycle0 or cycle1, same with other bam files i have looked at - and bamworkshop also indicates this - shows all frames available, but for the defined cycles, some frames are not included. Strange? or am i missing something?

This is actually a fairly common occurrence. BAMWorkshop II has the ability to list and display all frames independent of what sequence uses them while BAMWorkshop does not. This has always been, in my opinion, one of the biggest oversights in the design of BAMWorkshop. DLTCEP has a button to drop unused frames in the BAM editor window, and can check all BAMs in a game for several different "inconsistencies", including unused frames. Here is a list from my BG2EE game:


Spoiler


Edited by Sam., 02 April 2015 - 07:14 AM.

"Ok, I've just about had my FILL of riddle asking, quest assigning, insult throwing, pun hurling, hostage taking, iron mongering, smart-arsed fools, freaks, and felons that continually test my will, mettle, strength, intelligence, and most of all, patience! If you've got a straight answer ANYWHERE in that bent little head of yours, I want to hear it pretty damn quick or I'm going to take a large blunt object roughly the size of Elminster AND his hat, and stuff it lengthwise into a crevice of your being so seldom seen that even the denizens of the nine hells themselves wouldn't touch it with a twenty-foot rusty halberd! Have I MADE myself perfectly CLEAR?!"

--<CHARNAME> to Portalbendarwinden

--------------------

post-10485-0-15080600-1348188745.jpg
___________Old pen and paper modules of the 70s and 80s.___________

CA Forums CA Homepage


#9 fearless

fearless
  • Member
  • 79 posts

Donator

Posted 02 April 2015 - 08:14 AM

This assumes that the Frame Data for the first Frame Entry is ordered first in the BAM. While this may appear to be a logical thing to assume, I don't think it is necessarily guaranteed. It would be safer to loop through all the Frame Entries and take the lowest value of FrameDataOffset to be the actual start of the Frame Data section. Note that a Frame Entry with Width and/or Height of zero (0) need not necessarily have a reasonable FrameDataOffset (as it would have no real frame data) and so should be excluded from consideration of the lowest value of FrameDataOffset

 

I agree, it would be safer to loop through the existing structures and process the framelookup sequences whilst processing the cycles for example, which is what i have done. Too many oddities in those IE files it seems to assume anything for sure ;)


fearless

 

CM690 II Case, Corsair HX1000 PSU, Asus Maximus V Gene, Intel i5-3750K @ 4.2Ghz, Corsair 8GB DDR3, Asus DirectCU GTX-670, Samsung 256GB SSD, WD Caviar Black 2TB HDD x2

 

https://github.com/mrfearless