Jump to content


Photo

Lionheart-Extracting the Maps


  • Please log in to reply
117 replies to this topic

#1 leahnkain

leahnkain
  • Modder
  • 7649 posts

Posted 22 March 2008 - 05:56 PM

I have looked everywhere in Tokyo and I can't find a copy of Lionheart...it wasn?t a popular game at all in Japan. I am also looking for the temple district map, the Goblin Camp and Montaillou. I would really love to get the maps if anyone can extract them or make screenshots so a crude map can be made.(I can clean it up). When I am done, I can share a copy of the map if you need it. If anyone has a saved game that would be great as well.

Edited by leahnkain, 02 April 2008 - 10:20 PM.

Longing for the old pen and paper modules of the 70's and 80's. Experience AD&D's greatest adventures using the infinity engine: Visit our homepage at http://classicadventuresmod.com/


#2 Miloch

Miloch

    Barbarian

  • Modder
  • 6538 posts

Posted 29 March 2008 - 07:32 AM

I hacked around a little but can't find a way of extracting maps from this game. It doesn't have an override folder or BIFFs like the IE games. Instead, all the game data seems to be compiled in one massive (1.5 GB) data.dat file I can't even open with a hex editor. So it seems the only way of possibly doing it is to paste together a bunch of screenshots as Rabain suggested a couple years back. Aside from being a whole lot of work, you'd get junk on the maps (from other characters or your own, lighting effects, etc.) and you'd be able to see the "seams" unless you were extremely meticulous in pasting the images together.

I hacked around some more to try to find an editor or something. I came across this suggestion there is a development .exe of the Velocity Engine (used in this game as opposed to IE) but I see no evidence for that on the disks or in the installed folder. There is also suggestion the maps are JPGs, but if so, they're all compiled in that huge data file which I can't open (too big for application memory I suspect, even though I have 4 GB).

Infinity Engine Contributions
Aurora * BG1 NPC * BG1 Fixpack * Haiass * Infinity Animations * Level 1 NPCs * P5Tweaks
PnP Free Action * Thrown Hammers * Unique Containers * BG:EE * BGII:EE * IWD:EE
================================================================
Player & Modder Resources
BAM Batcher * Creature Lister * Creature Checker * Creature Fixer * Tutu/BGT Area Map & List * Tutu Mod List
================================================================
"Infinity turns out to be the opposite of what people say it is. It is not 'that which has nothing beyond itself' that is infinite, but 'that which always has something beyond itself'." -Aristotle


#3 WeeRLegion

WeeRLegion

    TFP : D

  • Member
  • 2925 posts

Posted 30 March 2008 - 06:24 AM

I used to have that game...
Hummm, IIRC, you can just unzip the main data folder.
And unzipping everything into a folder by the same name, and renaming the original file, well, you can just start editing... and ehh, images...
Well, no idea what format you'll find the maps in. x]

#4 Miloch

Miloch

    Barbarian

  • Modder
  • 6538 posts

Posted 30 March 2008 - 08:28 AM

Well, no idea what format you'll find the maps in. x]

I looked at this some more. You can get data files out of data.dat, but you need a high-grade extractor - I don't think zip or rar will handle it. I managed to get the subfiles and look at them.

The maps appear to be in a proprietary .frm16 format. I peered at it with a hex editor... it vaguely resembles the BMP format or maybe the MOS format the IE area graphics are in. Each area is accompanied by an "intermediate" .frm16 and a .way file. These might be the equivalent of the IE WED and ARE files, but I haven't really looked at them yet.

This is a summary of the BMP format:
Header (54 bytes)
Offset  Size  Description
0x00	2	 Signature ('BM')
0x02	8	 File size in bytes
0x0a	4	 Data offset (36)
0x0e	4	 Info header size (40)
0x12	4	 Width in pixels
0x16	4	 Height in pixels
0x1a	2	 Planes (1)
0x1c	2	 Bit count
0x1e	4	 Compression (0)
0x22	4	 Data size
0x26	4	 Horizontal pixels/meter
0x2a	4	 Vertical pixels/meter
0x2e	4	 Colors used (0)
0x32	4	 Colors important (0)
0x36	x	 Image data (where x = data size)
And this is my guesswork on the FRM16 format:
Header (28 bytes)
Offset  Size  Description
0x00	2	 Signature (1032)
0x06	2	 Width in pixels (portraits = 90, areas = 779)
0x08	2	 Height in pixels (portraits = 111, areas = 330)
0x0a	2	 Unknown (variably 0, 1182, 1565, 2624, 3711, etc.)
0x0c	2	 Unknown (64 or 68)
0x10	4	 Data size (areas = 515496-514140 = 1356/54C)
0x24	4	 Data size (redundant)
0x28	x	 Image data (where x = Data size)
y	   z	 Index data? (where y = file size - (data size + 28)
For frm16 areas, each element in the image data takes two bytes (same as 16-bit high-color BMP encoding). Since all areas are 779 * 330, the full data size is 257070 * 2 = 514140. If the area if physically smaller than this, blanks are padded with zeroes (black).

The index data seems to just follow a standard progression of integers that gets larger as the file size does. For example, a smaller file has 4, 10, 16, 22, 28, 36, 42, 67, 92, 98, 123, 131, 156, 181, 206, 214. Not sure what this means as it isn't even a standard progression. This is for a 12x16 icon though, so it could be related to the row data. Might also have something to do with the unknown at 0x0a - possibly some sort of palette data. I just discarded it since I didn't know what to do with it.

I wrote a little WeiDU snippet to convert the FRM16 areas to BMPs.
COPY ~sourcemap.frm16~ ~destmap.bmp~
  PATCH_IF (SOURCE_SIZE > 0x7dda7) THEN BEGIN
	WRITE_SHORT 0x0 19778 //Signature (BM)
	WRITE_LONG 0x2 514854 //File size (0x7db26)
	INSERT_BYTES 0x6 12
	WRITE_BYTE 0xa 54 //Data offset (0x36)
	WRITE_BYTE 0xe 40 //Info header size (0x28)
	INSERT_BYTES 0x14 2
	INSERT_BYTES 0x18 2
	WRITE_SHORT 0x1a 1 //Planes
	WRITE_SHORT 0x1c 16 //Bit count (0x10)
	INSERT_BYTES 0x20 2
	WRITE_LONG 0x26 0 //Horizontal resolution
	WRITE_LONG 0x2a 0 //Vertical resolution
	DELETE_BYTES 0x36 4
	READ_LONG 0x22 ds //Data size
	fs = ds + 0x36 //File size (data + header)
	rs = ds + 0x28 //Original size (data + old header)
	ns = SOURCE_SIZE - rs //Index size (old file size - original)
	DELETE_BYTES fs ns //Delete index
	WRITE_LONG 0x22 514800 //Data size (0x7daf0)
	FOR (r = 512636; r > 54; r -= 1558) BEGIN
	  INSERT_BYTES r 2
	END
  END
Anyway, this is what I ended up with for the Port District:
Posted Image

It's supposed to look something like this (from GameBanshee):
Posted Image
So my conversion is obviously a bit skewed, the palette is a bit off, and I had to flip it in PSP (the FRM16 row data is obviously reversed from BMPs). But I'm surprised I got a readable BMP at all from my low-level hackery. Maybe I can refine the conversion routine, but it probably requires a techno-guru... it's a bit much for my poor orcish brain. I'm attaching the original frm16 file and my bmp if anyone wants to try to figure more about the format.

Attached File  portdist.bmp   754.15K   511 downloadsAttached File  portdist.frm16.txt   503.41K   427 downloads
You can drop the .txt extension from the .frm16 file. It is not that readable in a text editor unless you have one that also supports hex (PSPad for example).

I know you didn't ask for the Port District, leahnkain, but it looked like one of the easier maps to try this on. If we can figure out the frm16 format, we can probably come up with a routine to convert any and all areas - maybe the animations too.

Edit: WeiDU updated (see below for results)

Edited by Miloch, 02 April 2008 - 11:27 AM.

Infinity Engine Contributions
Aurora * BG1 NPC * BG1 Fixpack * Haiass * Infinity Animations * Level 1 NPCs * P5Tweaks
PnP Free Action * Thrown Hammers * Unique Containers * BG:EE * BGII:EE * IWD:EE
================================================================
Player & Modder Resources
BAM Batcher * Creature Lister * Creature Checker * Creature Fixer * Tutu/BGT Area Map & List * Tutu Mod List
================================================================
"Infinity turns out to be the opposite of what people say it is. It is not 'that which has nothing beyond itself' that is infinite, but 'that which always has something beyond itself'." -Aristotle


#5 leahnkain

leahnkain
  • Modder
  • 7649 posts

Posted 31 March 2008 - 04:34 PM

Sorry to take so long to reply. Miloch do you think a frm16 to .bmp for fallout can be converted to Lionheart? It is the only program I have been able to find on the web. I downloaded a demo of the game and I have had no luck. The best I was able to do was using WinRAR to create an archive file consisting solely of the data.dat file in [default path] C:\Program Files\Black Isle\Lionheart. I used .rar as the file format for the archive (.zip could work too, I didn't test it though). After I got the .rar package,I opened it with WinRAR and double-clicked on the data.dat file inside, it started loading for preview as it is with all .rar packages, and this time it really opened the files for me to see (The loading took a few minutes). Then I selected them all and extracted them to another folder. I found with the demo version there are frm16 files and they have been impossible for me to figure out how to open. I made the port district into a map a year ago. (Thanks to Rabain's screenshots) If anyone can get me the screenshots I can always make it into a map. I will happily share the maps. I think Lionheart can offer a lot of areas and animations which a lot of modders would love to get there hands on. If we can all work together to figure this out it would be a great asset for the community.

Edited by leahnkain, 31 March 2008 - 05:08 PM.

Longing for the old pen and paper modules of the 70's and 80's. Experience AD&D's greatest adventures using the infinity engine: Visit our homepage at http://classicadventuresmod.com/


#6 Miloch

Miloch

    Barbarian

  • Modder
  • 6538 posts

Posted 02 April 2008 - 11:25 AM

Sorry to take so long to reply. Miloch do you think a frm16 to .bmp for fallout can be converted to Lionheart?

I tried one of these converters and got about 36 kinds of rubbish. Tried a few more - no luck with them either. It looks like the Fallout .frm format is more for animations, kind of like .bam files for IE so I don't think it's at all similar to frm16.

On the other hand, I updated my WeiDU snippet to insert a couple extra bytes after each row and it worked like a charm. After I flip it to reverse the rows (BMPs are upside-down I found out), I get this for the Temple District:
Posted Image
Which is pretty good, compared with the GameBanshee map:
Posted Image
Another problem with screen shots is that you'll have a lot of dark space where you can't see, like above. This is in addition to the other issues I mentioned with screen shots.

So we just need to figure out how to fix the palette and then see how the game translates the smaller maps to bigger ones. I'm not quite sure how that works, but BG seems to do the same thing for .mos files, because they aren't nearly as large in the files as they are on screen.

In the meantime, maybe a moderator can move this to IE Modding Help or perhaps Layers of Reality (though it's probably more technical than artistic). It'll probably get more serious interest there than as a Recreation Room topic, and I'm not sure I can figure this all out by myself.

Infinity Engine Contributions
Aurora * BG1 NPC * BG1 Fixpack * Haiass * Infinity Animations * Level 1 NPCs * P5Tweaks
PnP Free Action * Thrown Hammers * Unique Containers * BG:EE * BGII:EE * IWD:EE
================================================================
Player & Modder Resources
BAM Batcher * Creature Lister * Creature Checker * Creature Fixer * Tutu/BGT Area Map & List * Tutu Mod List
================================================================
"Infinity turns out to be the opposite of what people say it is. It is not 'that which has nothing beyond itself' that is infinite, but 'that which always has something beyond itself'." -Aristotle


#7 Kaeloree

Kaeloree

    Head Molder

  • Administrator
  • 9195 posts

Posted 02 April 2008 - 12:06 PM

Moved to IE Modding Help--it might be worth editing the topic title as well, to be more specific about what the thread is about :)

#8 leahnkain

leahnkain
  • Modder
  • 7649 posts

Posted 02 April 2008 - 03:41 PM

I can always repaint the buildings. It will take me a lot of time but I can give it a shot. How clear are the buildings? Are they really fuzzy and lacking detail?

Longing for the old pen and paper modules of the 70's and 80's. Experience AD&D's greatest adventures using the infinity engine: Visit our homepage at http://classicadventuresmod.com/


#9 Tassadar88

Tassadar88

    Templar in Flames

  • Member
  • 1302 posts

Posted 02 April 2008 - 10:28 PM

Well, I have the game at home now (bargain binīs best buys) and am willing to do the raw extracting and to upload it once you are happy with the translation formula if you want Miloch :)
The Mind is its own place and in itself - can make a Heaven of Hell, a Hell of Heaven. -John Milton, Paradise lost

#10 Miloch

Miloch

    Barbarian

  • Modder
  • 6538 posts

Posted 03 April 2008 - 06:41 AM

Well, I have the game at home now (bargain binīs best buys) and am willing to do the raw extracting and to upload it once you are happy with the translation formula if you want Miloch :)

I think I have all the source files - I need help actually with the translation formula. I'm thinking now that these .frm16 files are the mini-maps, like .mos files, and maybe the bigger areas are stored in the .way files. But even those don't really look big enough, and they don't seem to be compressed or anything. We really need a 'VESDP' (Velocity Engine Structures Description Project) like what they did for the IESDP.

Screenshots may still be an option, if you want to take some of those. I just think figuring out the format might be quicker in the long run, especially if we're going to be converting more than one of these areas.

Infinity Engine Contributions
Aurora * BG1 NPC * BG1 Fixpack * Haiass * Infinity Animations * Level 1 NPCs * P5Tweaks
PnP Free Action * Thrown Hammers * Unique Containers * BG:EE * BGII:EE * IWD:EE
================================================================
Player & Modder Resources
BAM Batcher * Creature Lister * Creature Checker * Creature Fixer * Tutu/BGT Area Map & List * Tutu Mod List
================================================================
"Infinity turns out to be the opposite of what people say it is. It is not 'that which has nothing beyond itself' that is infinite, but 'that which always has something beyond itself'." -Aristotle


#11 leahnkain

leahnkain
  • Modder
  • 7649 posts

Posted 03 April 2008 - 07:21 AM

I agree Miloch's idea is much better. I have worked on making a map from screenshots, it takes a lot of time. I think a lot of modders could benefit from these areas. Hopefully more people can show support.

Longing for the old pen and paper modules of the 70's and 80's. Experience AD&D's greatest adventures using the infinity engine: Visit our homepage at http://classicadventuresmod.com/


#12 Tassadar88

Tassadar88

    Templar in Flames

  • Member
  • 1302 posts

Posted 03 April 2008 - 11:18 AM

Well, I have the game at home now (bargain binīs best buys) and am willing to do the raw extracting and to upload it once you are happy with the translation formula if you want Miloch :)

I think I have all the source files - I need help actually with the translation formula.
...
I just think figuring out the format might be quicker in the long run, especially if we're going to be converting more than one of these areas.

Canīt help there -I only half understand the code you wrote and I would have no idea of what to change to get it right, but once you figure it out, I will gladly help with the mindless tedium of conversion/sorting/uploading :)
The Mind is its own place and in itself - can make a Heaven of Hell, a Hell of Heaven. -John Milton, Paradise lost

#13 leahnkain

leahnkain
  • Modder
  • 7649 posts

Posted 03 April 2008 - 05:40 PM

Well, I have the game at home now (bargain binīs best buys) and am willing to do the raw extracting and to upload it once you are happy with the translation formula if you want Miloch :)


Tassadar88 would it be possible for you to upload the .way files and the frm16 so the other modders and myself who don't have the game can help out?

Longing for the old pen and paper modules of the 70's and 80's. Experience AD&D's greatest adventures using the infinity engine: Visit our homepage at http://classicadventuresmod.com/


#14 Tassadar88

Tassadar88

    Templar in Flames

  • Member
  • 1302 posts

Posted 03 April 2008 - 11:42 PM

Well, I have the game at home now (bargain binīs best buys) and am willing to do the raw extracting and to upload it once you are happy with the translation formula if you want Miloch :)


Tassadar88 would it be possible for you to upload the .way files and the frm16 so the other modders and myself who don't have the game can help out?

I will get home sometime late tomorrow, then Iīll upload it. I will only give the DL access upon PM request to keep the "ZOMGWTF!!! U R PIRATZORRING TEH GAME!!" feel of it to the lowest level possible. (besides, if I leave out the rest of the game content, it will be unplayable anyway so hopefully noone will mind :unsure: )
The Mind is its own place and in itself - can make a Heaven of Hell, a Hell of Heaven. -John Milton, Paradise lost

#15 Miloch

Miloch

    Barbarian

  • Modder
  • 6538 posts

Posted 06 April 2008 - 08:40 AM

This archive contains the data for the Temple District - an unchanged .frm16 minimap, a .bmp I converted from it, an "intermediate data" .frm16 and a .way file. I haven't figured out what those last two are yet, so that is really where we need help. If people can take a look at the files with a hex editor and try to figure out at least the data header, IESDP-style, it'll give us a huge boost.
Attached File  tempdist.rar   619.2K   386 downloads
Edit: Here are some more samples, by request. Includes same four files as above. Montaillou:
Attached File  montaill.rar   638.19K   358 downloadsPosted Image

Goblin Camp:
Attached File  goblcamp.rar   349.13K   345 downloadsPosted Image
This includes one of the goblin house interiors for comparison of a smaller area:
Posted Image

Here is an archive with a few sample portraits and textures, for further comparison.
Attached File  samples.rar   57.18K   316 downloads
And finally, here is my frm16>bmp converter in WeiDU mod format, along with another sample map. It works on all the minimaps, but gives garbage bitmaps for portraits etc.
Attached File  lionmaps.rar   497.76K   406 downloads

Edited by Miloch, 06 April 2008 - 07:25 PM.

Infinity Engine Contributions
Aurora * BG1 NPC * BG1 Fixpack * Haiass * Infinity Animations * Level 1 NPCs * P5Tweaks
PnP Free Action * Thrown Hammers * Unique Containers * BG:EE * BGII:EE * IWD:EE
================================================================
Player & Modder Resources
BAM Batcher * Creature Lister * Creature Checker * Creature Fixer * Tutu/BGT Area Map & List * Tutu Mod List
================================================================
"Infinity turns out to be the opposite of what people say it is. It is not 'that which has nothing beyond itself' that is infinite, but 'that which always has something beyond itself'." -Aristotle


#16 leahnkain

leahnkain
  • Modder
  • 7649 posts

Posted 06 April 2008 - 10:25 PM

I am going to see what I can do today. Thanks for uploading everything.

Longing for the old pen and paper modules of the 70's and 80's. Experience AD&D's greatest adventures using the infinity engine: Visit our homepage at http://classicadventuresmod.com/


#17 Tassadar88

Tassadar88

    Templar in Flames

  • Member
  • 1302 posts

Posted 08 April 2008 - 03:55 AM

Ok, so I have packed the 320 Mb of relevant data from the datafile here. It is in a 45 Mb self extracting archive now and I am trying my luck with extracting the frm16 as well :)
The Mind is its own place and in itself - can make a Heaven of Hell, a Hell of Heaven. -John Milton, Paradise lost

#18 leahnkain

leahnkain
  • Modder
  • 7649 posts

Posted 11 April 2008 - 03:46 AM

Has anyone had any luck yet?

Longing for the old pen and paper modules of the 70's and 80's. Experience AD&D's greatest adventures using the infinity engine: Visit our homepage at http://classicadventuresmod.com/


#19 Tassadar88

Tassadar88

    Templar in Flames

  • Member
  • 1302 posts

Posted 11 April 2008 - 06:35 AM

I tried and failed :wall:
The Mind is its own place and in itself - can make a Heaven of Hell, a Hell of Heaven. -John Milton, Paradise lost

#20 leahnkain

leahnkain
  • Modder
  • 7649 posts

Posted 11 April 2008 - 07:02 AM

So far I have been failing as well. I hope more modders will give it a shot. These maps would look great in BG.

Edited by leahnkain, 11 April 2008 - 07:02 AM.

Longing for the old pen and paper modules of the 70's and 80's. Experience AD&D's greatest adventures using the infinity engine: Visit our homepage at http://classicadventuresmod.com/