Jump to content


Photo

TileConv - A MOS/TIS Compressor

TIS MOS packer unpacker

52 replies to this topic

#1 Argent77

Argent77
  • Member
  • 734 posts

Posted 30 August 2014 - 05:24 AM

File Name: TileConv - A MOS/TIS Compressor
File Submitter: Argent77
File Submitted: 30 Aug 2014
File Category: IE Modding Tools

TileConv is an advanced tiled image compressor and decompressor for MOS and TIS files used in Infinity Engine games (such as Baldur's Gate, Icewind Dale or Planescape Torment). It uses modern compression algorithms to achieve great compression ratios without noticeable degradation in quality. The average compression size lies around 20-30% of the original size and will never exceed 40% when using default compression options.

The tool can convert between the following formats:
  • TIS files will be automatically compressed into the TBC format.
  • MOS files will be automatically compressed into the MBC format.
  • TBC files will be automatically decompressed into the TIS format.
  • MBC files will be automatically decompressed into the (compressed or uncompressed) MOS format.
  • TIZ files will be automatically decompressed into the TIS format.
  • MOZ files will be automatically decompressed into the (compressed or uncompressed) MOS format.
TileConv is aimed at modders who wish to provide their own TIS or MOS files without having large downloads. The package contains the source code as well as binaries for Windows, Mac OS X and Linux. The latest source code is also available on GitHub. A detailed description of the TBC and MBC formats can be found here.
 
This package also contains a collection of WeiDU functions to deal with tileconv executables for multiple platforms and architectures, including a detailed description and example code.
 
Usage
Spoiler

Note: The tileconv executables have been compressed to reduce file size. Some overzealous antivirus tools may trigger false alarms because of this.

Click here to download this file

#2 Argent77

Argent77
  • Member
  • 734 posts

Posted 30 August 2014 - 08:29 AM

In contrast to the old and known tispack which uses jpeg compression to reduce size, tileconv applies methods originally developed to compress textures.

The main advantage of tileconv over tispack is the ability to preserve complex details much more faithfully. You can compare the respective quality in the following images, taken from various TIS files. For simplicity I have labeled the image with numbers (1: original image from the game, 2: compressed with tileconv, 3: compressed with tispack using default quality, 4: compressed with tispack using a quality that produces files of comparable size).

This is a section of the BG2 Harper building:

Spoiler


The entrance to the Umar Inn:
Spoiler


More examples in the next post.


Edited by Argent77, 30 August 2014 - 08:30 AM.


#3 Argent77

Argent77
  • Member
  • 734 posts

Posted 30 August 2014 - 08:31 AM

The Oasis on the way to Amkethran:

Spoiler


Underdark area in Icewind Dale 2:
Spoiler


More examples in the next post.



#4 Argent77

Argent77
  • Member
  • 734 posts

Posted 30 August 2014 - 08:32 AM

Underground cave in Icewind Dale:

Spoiler


Another section from the same underground cave in Icewind Dale:
Spoiler


More examples in the next post.



#5 Argent77

Argent77
  • Member
  • 734 posts

Posted 30 August 2014 - 08:33 AM

And last but not least, inside of the Great Foundry in Planescape Torment:

Spoiler


Edited by Argent77, 30 August 2014 - 08:34 AM.


#6 Yovaneth

Yovaneth

    The newly-appointed Master Builder of Baldur's Gate

  • Modder
  • 3012 posts

Posted 31 August 2014 - 01:57 AM

Just as I was starting to get annoyed with tispack's poor performance with details... :clap: :D



#7 The Imp

The Imp

    Not good, see EVIL is better. You LIVE.

  • Member
  • 3908 posts

Posted 31 August 2014 - 03:00 AM

How much is this really going to effect the size of the archive, realistically ? I would want an example if you got one.
This especially if we cut and paste the .tis files directly from the mod folder to the game folders filing system... so no copy pasting.


Edited by The Imp, 31 August 2014 - 03:05 AM.

Yep, Jarno Mikkola. my Mega Mod FAQ. Use of the BWS, and how to use it(scroll down that post a bit). 
OK, desert dweller, welcome to the sanity, you are free to search for the limit, it's out there, we drew it in the sand. Ouh, actually it was still snow then.. but anyways.


#8 Argent77

Argent77
  • Member
  • 734 posts

Posted 31 August 2014 - 05:22 AM

Compression size varies a lot depending on the sources. During my tests I have compressed all MOS and TIS files from BG, BG2, IWD, IWD2 and PST. The Compression size was as low as 3% and as high as 39%. Because of the algorithm used to compress files, it will never exceed 40%. The only downside at the moment is the slow encoding and decoding speed, but I'm already trying to solve this issue. (Edit: Already solved.)

@Jarno You want examples? Here they are:

Map name                              TIS (in bytes)  TBC (bytes / % of TIS)  Zipped TIS (bytes / % of TIS)  Zipped TBC (bytes / % of TIS)
Athkatla Docks (AR0300)                     26982424         8917965 / 33.0%               18711363 / 69.3%                8916762 / 33.0%
Council of Six Interior (AR1002)             2688024          420658 / 15.6%                 795337 / 29.5%                 411593 / 15.6%
Druid Grove (AR1900)                        23490584         8635792 / 36.7%               19773250 / 84.1%                8638342 / 36.7%
Trademeet (AR2000)                          18595864         6904333 / 37.1%               16254220 / 87.4%                6906219 / 37.1%
Watcher's Keep (AR3000)                     10388504         3517325 / 33.8%                7847019 / 75.5%                3517915 / 33.8%
Throne of Bhaal (AR6200)                     6799384         1885648 / 27.7%                3101246 / 45.6%                1885415 / 27.7%
Wilderness outside Saradush (AR6400)         5990424         1438101 / 24.0%                5202229 / 86.8%                1438537 / 24.0%

As you can see, TBC files can't be compressed much more because they're using a similar compression technique internally. But there are tricks to increase the compression ratio even further.

I've also tested it on my own mod and the overall file size of the 7zip archive dropped from 27 MB to 19 MB. The mod contains 14 MOS files and 5 TIS files with a total size of 34 MB.


Edited by Argent77, 01 September 2014 - 05:01 AM.


#9 The Imp

The Imp

    Not good, see EVIL is better. You LIVE.

  • Member
  • 3908 posts

Posted 31 August 2014 - 07:28 AM

Compression size varies a lot depending on the sources. D....

Obviously. Also if we use different archive tool, we might get different results as well. But I don't know if the non -zip archive tools(.rar, .7z, ) lessen the quality of the picture, but that seems doubtful.
Thanks for the example. Now, then... :devil: ... see the .tbc looses it's efficiency partially little to the ideal .tis packaging where the mod folder is emptied of the excess files, aka the .tis files are moved to the game folder instead of copying them over. So the end result taken from the last line(ar6400):
Download space in .zip archive file size 5990424:  5202229 vs. 1438537 (bytes, duh), difference per original file: ~62.8% . Or in other words the .tbc is about 3 times faster to download.
Difference on the final install folder with the archive: (5990424+5202229) vs. (5990424+1438101+1438537) => 11192653 vs. 8867062 => 186.8% vs 148.0 % =>
So the complete difference is: 38.8% for the space usage of the .tbc files.
The fact that the archive can be then deleted from the computer, brings a grim note here. 0 vs. 1438101 => -24.0% :devil:

All this says that it might be better off to, for me in my opinion using the original .tis files, but that's cause I don't mind the extra download size... but whatever... :D

And now, I thank you for the entertainment.


Edited by The Imp, 31 August 2014 - 08:01 AM.

Yep, Jarno Mikkola. my Mega Mod FAQ. Use of the BWS, and how to use it(scroll down that post a bit). 
OK, desert dweller, welcome to the sanity, you are free to search for the limit, it's out there, we drew it in the sand. Ouh, actually it was still snow then.. but anyways.


#10 Daulmakan

Daulmakan

    Comfortably numb

  • Member
  • 1061 posts

Posted 31 August 2014 - 08:02 AM

Seems great, Argent. Kudos. :new_thumbs:



#11 Argent77

Argent77
  • Member
  • 734 posts

Posted 01 September 2014 - 04:43 AM

Version 0.2 has been released!

 

It provides a number of new tweaking options, and conversion speed has been greatly increased, depending on the number of available cores of your CPU. Additionally I have tweaked several quality features to increase speed even further. On my system (Intel Core i7, Quad-Core) conversion times have been decreased to 20%.



#12 -ac-

-ac-
  • Guest

Posted 01 September 2014 - 07:01 AM

58.1 MB Original MOS files
18.7 MB PNG files
18.0 MB webp (lossless) files
16.4 MB Zipped MOS files
12.8 MB 7zipped MOS files
------------------
05.3 MB 7zipped MBC (type 0x101) files
02.7 MB webp (lossy) files

#13 Argent77

Argent77
  • Member
  • 734 posts

Posted 01 September 2014 - 07:39 AM

webp looks definitely interesting, provided the lossy version meets certain quality standards. But it will probably lose some of its efficiency when applied to tile based graphics formats.
 



#14 Sam.

Sam.
  • Modder
  • 935 posts

Posted 01 September 2014 - 12:57 PM

1: BC1/DXT1 (Default)
2: BC2/DXT3
3: BC3/DXT5

 

Are these the underlying compression algorithms used in the .PVR format in the Enhanced Editions?


"Ok, I've just about had my FILL of riddle asking, quest assigning, insult throwing, pun hurling, hostage taking, iron mongering, smart-arsed fools, freaks, and felons that continually test my will, mettle, strength, intelligence, and most of all, patience! If you've got a straight answer ANYWHERE in that bent little head of yours, I want to hear it pretty damn quick or I'm going to take a large blunt object roughly the size of Elminster AND his hat, and stuff it lengthwise into a crevice of your being so seldom seen that even the denizens of the nine hells themselves wouldn't touch it with a twenty-foot rusty halberd! Have I MADE myself perfectly CLEAR?!"

-- <charname> to Portalbendarwinden

--------------------

Posted Image
___________Old pen and paper modules of the 70s and 80s.___________

CA Forums CA Homepage


#15 Argent77

Argent77
  • Member
  • 734 posts

Posted 01 September 2014 - 01:41 PM

Yes, they are. At least in the Windows and Mac versions. The PVR files in mobile versions are using some proprietary texture compression format.
 



#16 The Imp

The Imp

    Not good, see EVIL is better. You LIVE.

  • Member
  • 3908 posts

Posted 01 September 2014 - 01:48 PM

Yes, they are. At least in the Windows and Mac versions. The PVR files in mobile versions are using some proprietary texture compression format.

*Throws the Sam.'s wrench to Argen77* -Are you sure those are not copyrighted ? & They are ok with it ? Just asking cause I would hate to see you & the community to loose ...


Yep, Jarno Mikkola. my Mega Mod FAQ. Use of the BWS, and how to use it(scroll down that post a bit). 
OK, desert dweller, welcome to the sanity, you are free to search for the limit, it's out there, we drew it in the sand. Ouh, actually it was still snow then.. but anyways.


#17 Argent77

Argent77
  • Member
  • 734 posts

Posted 01 September 2014 - 02:04 PM

I'm NOT touching the (proprietary) PVR container format in any way. The library to encode and decode BCn/DXTn compressed pixels is open source.



#18 The Imp

The Imp

    Not good, see EVIL is better. You LIVE.

  • Member
  • 3908 posts

Posted 01 September 2014 - 02:10 PM

I'm NOT touching the (proprietary) PVR container format in any way. The library to encode and decode BCn/DXTn compressed pixels is open source.

*and the boomerang formed wrench goes around Argent77's head and circulates back to the wonderfully surprised Imp's face, that's wickedly smiling to find it's friend coming at it*

Auch, erhm... THANKS. :D


Edited by The Imp, 01 September 2014 - 02:17 PM.

Yep, Jarno Mikkola. my Mega Mod FAQ. Use of the BWS, and how to use it(scroll down that post a bit). 
OK, desert dweller, welcome to the sanity, you are free to search for the limit, it's out there, we drew it in the sand. Ouh, actually it was still snow then.. but anyways.


#19 Solaufein

Solaufein

    No fate but what we make

  • Member
  • 262 posts

Posted 01 September 2014 - 11:17 PM

That is lovely. I have an 800MB mod and most of those megabytes is TIS files. So this would be helpful. Where is the instructions on how to use it?


Her lips were red, her looks were free
Her locks were as yellow as gold
Her skin was white as leprosy.
The Nightmare Life-in-Death was she,
Who thicks man's blood with cold.

Samuel Taylor Coleridge

http://teambg.net
The Undying
 


#20 Argent77

Argent77
  • Member
  • 734 posts

Posted 02 September 2014 - 02:21 AM

You can find the usage described in the included README and also in the first post of this thread. It should be pretty self-explanatory. The tool detects the right format automatically and converts it into its counterpart (i.e. TIS↔TBC and MOS↔MBC). For example, calling "tileconv test1.tis test2.mos test3.tbc test4.mbc" will result in the files test1.tbc, test2.mbc, test3.tis and test4.mos.

TileConv's defaults should be sufficient for the majority of cases, so you'll rarely need to specify any of the available options (except maybe option -z to produce compressed MOS files). Do you have any specific questions?
 





Reply to this topic



  



Also tagged with one or more of these keywords: TIS, MOS, packer, unpacker