Jump to content


Photo

IMProved Generalized Biffing.

bif bws megamod

  • Please log in to reply
12 replies to this topic

#1 The Imp

The Imp

    Not good, see EVIL is better. You'll LIVE.

  • Member
  • 5015 posts

Posted 18 December 2015 - 03:08 AM

... or the moving away from The Bigg's mod to a custom one.

Well, I meant to make a PM topic about this, and post it to a few people, but then had the notion that it might as well be good to take imput from everyone, here.
This will be a long ramble, aren't all my topics :P , but there aren't that many of them, so read it at your own leisure la... 'K. .. didn't write this post in one go. So it's a proper ramble.

A bit of history. Yes, the ramble starts from here... I try to keep this straight, but if you don't remember it all, I don't, you are likely to unintentionally glance stuff and miss important notions. So try keep up.
In the past, the game and mods have utilized the biffing to make the game run faster, and so it's internal fact that the Mega Moding needs to take it into account, like for example all the big mods use biffing today, list that I have is: BGT-weidu, TDD, SoS, NEJ, CtB, Dark Horizon, Bonehill, NTotSC and so forth. There's a few more.
Now, look at the list and notice that the current BWS removes most of the Biffing from those mods if you install the Lollorian's trimmed Fixpack, aka the BiG World Fixpack upwards from v15. The reasoning is good, as it makes the mod installs generally better, by allowing them to be reinstallable, it saves hard drive space, it removes time expenditure that would go into frivolous biffing and it delivers cookies. Well, ok, it doesn't do the last thing. The bad side of that is that the game then really rreeaallllyy needs to be biffed at the end of the install to speed up the games starting process and running speed.
Now, in the past, this has been resolved by 'forcing' the installation of the tool that does this. In the prime of BWP which was years ago, when the Lollorians Trimpack was still in it's none existing stage, that then got consumed into the Fixpack too, the first mod was the END_BIFF.bat, but that had the humongous problem that it couldn't even be uninstalled in a strait chain weidu uninstall command, as it utilized the old .bat command, which can't restore the removed content.

That was partially solved when The Bigg then added a weidu command that can handle uninstall without .bat's etc and came up with the Generalized Biffing mod. You can actually see in that that his opinion was split there too, as he programmed two distinct forms to the component. And the .tra file reflects this:
~Would you like to biff only wav&tis files, or all files found?
0] only wav&tis&bam (recommended by The Bigg and other Small World Dudes)
1] all files (recommended by the Big World Dudes)~
The zero biffs just the media files, and the other is the only one that the BWS today uses, biffing everything.
It has a small "flaw", at least in the eyes of the BWS install, that doesn't allow the use of both of these modes, as they are not strictly component based, but inside the same component.
Added to that, the BWS today kinda needs to force biffing. And the only other possibility is still the END_BIFF.bat
I propose a small change that will remove the worst parts of that. This is still tested only a limited amount of times, if you can call one successful run a limited amount. :devil:
Spoiler

Edited by The Imp, 20 December 2015 - 01:49 AM.

Yep, Jarno Mikkola. my Mega Mod FAQ. Use of the BWS, and how to use it(scroll down that post a bit). 
OK, desert dweller, welcome to the sanity, you are free to search for the limit, it's out there, we drew it in the sand. Ouh, actually it was still snow then.. but anyways.


#2 The Imp

The Imp

    Not good, see EVIL is better. You'll LIVE.

  • Member
  • 5015 posts

Posted 18 December 2015 - 03:12 AM

I'll place a tactical second post here, so I can edit the first without loosing to unimportant stuff, by placing it here.

And in case it's not obvious, the topic title and the mods name starting with the IMP is a joke. :ph34r: As if  I would approve anything these evil talons type.


Edited by The Imp, 18 December 2015 - 03:23 AM.

Yep, Jarno Mikkola. my Mega Mod FAQ. Use of the BWS, and how to use it(scroll down that post a bit). 
OK, desert dweller, welcome to the sanity, you are free to search for the limit, it's out there, we drew it in the sand. Ouh, actually it was still snow then.. but anyways.


#3 Almateria

Almateria

    most garbage person

  • Modder
  • 945 posts

Posted 18 December 2015 - 03:47 AM

lol if you dont put your name in everything you create

way to let sid meier down



#4 Roxanne

Roxanne

    Modder

  • Member
  • 3564 posts

Posted 18 December 2015 - 04:12 AM

Some feedback from a player, tester and modder (trying to keep those aspects apart)

 

1. We do not need to discuss that biffing (#1 generalized biffing) is absolute MUST for any modded game of some significant size (loading times, in-game delay, size on your computer etc....)

2. Biffing was introduced already when most users still did their installations mostly by hand (before BWP/BWS) and that may explain the two existing options.

3. In the very beginning (almost a decade ago) I did option 0 biffing, mostly out of fear for having to reinstall the complete game, I think the option is more or less historic - with all the experience we have today I would advocate to forget about it completely. Do option 1 if you do any biffing (if not just blame yourself for lame gameplay).

4. The question of why and when you should biff the installation is more related to the question of which/why/when do you anticipate to UNINSTALL MODS.

5. From the point of just a player using BWP/BWS to do my install, I see no real reason to ever need to uninstall something. Regardless of having a biffed or non-biffed game, you spend a lot of time and risk a messed installatiion if you try to repair any installation this way. Especially if you have some large game tweak mods near the end or your installation. If you really encounter problems with a file or mod, always try to find the older version of the file(s) in the respective backup folder and restore it to override.

6. With BWP/BWS it is much easier today to just delete a messed installation and let the tools do a new one (all you need is the clean backup you created when first running one of the tools and the tool itself). Rather spend your time in the initial setup and selection of your install, rather than try to repair in reverse.

7. The only real reason to have to constantly install/re-install stuff on a running installation is if you develop mods yourself or you are beta-testing one or more. In this case, the best way for me is to install those unstable mods at the very end (even after biffing) and modify/update/uninstall them easily when needed.


The Sandrah Saga

another piece of *buggy, cheesy, unbalanced junk*

 


#5 The Imp

The Imp

    Not good, see EVIL is better. You'll LIVE.

  • Member
  • 5015 posts

Posted 18 December 2015 - 04:42 AM

Roxanne, have you ever installed the NPC component of the Level 1 NPCs mod ? After installing the first 2 out of the 300 other mods with BWS, and went to bed thinking that perhaps you could install the custom mods by hand and then use IGB. Or never having used the BWS and then having to choose the widescreen mods resolution option when you don't know what it looks like in your monitor when you are 70+ years ? It's a havoc to think about all the stumbling things you can fall into .. if you just, ahh cheese, there's always the reinstall option.

 

@Almateria: Well, I can't change my compilation mods name anymore, as the "Impilation" title is already implemented to the folder names structure in the .tp2 file. :P


Yep, Jarno Mikkola. my Mega Mod FAQ. Use of the BWS, and how to use it(scroll down that post a bit). 
OK, desert dweller, welcome to the sanity, you are free to search for the limit, it's out there, we drew it in the sand. Ouh, actually it was still snow then.. but anyways.


#6 Roxanne

Roxanne

    Modder

  • Member
  • 3564 posts

Posted 18 December 2015 - 04:47 AM

Roxanne, have you ever installed the NPC component of the Level 1 NPCs mod ? After installing the first 2 out of the 300 other mods with BWS, and went to bed thinking that perhaps you could install the custom mods by hand and then use IGB. Or never having used the BWS and then having to choose the widescreen mods resolution option when you don't know what it looks like in your monitor when you are 70+ years ? It's a havoc to think about all the stumbling things you can fall into .. if you just, ahh cheese, there's always the reinstall option.

Caught me - I NEVER EVER install Level 1 NPCs and I NEVER use widescreen mod because i know how it looks for my 70+ years eyes. And since I NEVER EVER WILL, I cannot comment any further.

 

@Almateria: Well, I can't change my compilation mods name anymore, as the "Impilation" title is already implemented to the folder names structure in the .tp2 file. :P

Edited by Roxanne, 18 December 2015 - 04:48 AM.

The Sandrah Saga

another piece of *buggy, cheesy, unbalanced junk*

 


#7 agb1

agb1
  • Member
  • 1623 posts

Posted 18 December 2015 - 05:58 AM

By the by, BWS now installs both Level1NPCs (just the kit change components, not the core tweaks) and NPC_EE after generalized biffing to allow for "oops, didn't want that" partial reinstalls.

I think that Imp is on the right track in questioning which smaller tweaks a user might change his or her mind about, and if we can make small install changes easier without a full reinstall (even if fully automated, it is still time consuming).

Another way to look at this: could WeiDU be improved such that it can selectively extract what it needs from the biffs instead of undoing the entire generalized biffing? Another way of saying it: could we make it seem to WeiDU that generalized biffing actually installed a different component for each file that is generally-biffed, instead of a single gen-biff component covering all files? This would allow WeiDU to unwind biffing on a per file basis when unwinding changes.

Even better would be if biffs could be made transparent to WeiDU so it just modifies the files in the biffs directly without having to unwind.

Of course this wouldn't fully solve the problem because some tweak mods (SCS for example) also change many files in a single component, so the unwinding time is still long even if you want to unwind far enough to change something before SCS. To help with this, we might be able to move more of the small (fewer files modified) tweaks after the big tweaks.

Maybe we could split gen biff component 1 into several components for different types of resource files. Items and creatures and spells and scripts each in separate biffs.

Edited by agb1, 18 December 2015 - 06:18 AM.

BiG World Fixpack (community collection of mod fixes and compatibility patches, with user-friendly cross-platform script)

 

BiG World Setup (tool to automate best-practice installation of Infinity Engine mods on Windows, with conflict analysis)

Latest version:    https://bitbucket.or.../get/master.zip


#8 Roxanne

Roxanne

    Modder

  • Member
  • 3564 posts

Posted 18 December 2015 - 06:23 AM

By the by, BWS now installs both Level1NPCs (just the kit change components, not the core tweaks) and NPC_EE after generalized biffing to allow for "oops, didn't want that" partial reinstalls.

I think that Imp is on the right track in questioning which smaller tweaks a user might change his or her mind about, and if we can make small install changes easier without a full reinstall (even if fully automated, it is still time consuming).

Another way to look at this: could WeiDU be improved such that it can selectively extract what it needs from the biffs instead of undoing the entire generalized biffing? Another way of saying it: could we make it seem to WeiDU that generalized biffing actually installed a different component for each file that is generally-biffed, instead of a single gen-biff component covering all files? This would allow WeiDU to unwind biffing on a per file basis when unwinding changes.

Even better would be if biffs could be made transparent to WeiDU so it just modifies the files in the biffs directly without having to unwind.

Of course this wouldn't fully solve the problem because some tweak mods (SCS for example) also change many files in a single component, so the unwinding time is still long even if you want to unwind far enough to change something before SCS. To help with this, we might be able to move more of the small (fewer files modified) tweaks after the big tweaks.

Maybe we could split gen biff component 1 into several components for different types of resource files. Items and creatures and spells and scripts tend to be targeted most by tweak mods.

I agree to your level1 npc reasoning - in some way this is not really a mod (I always only refer to the second part - the individual NPC tweak) because you may as well use shadow/ee keeper or NI to change your NPCs to whatever (Ajantis the totemic druid meets Minsc the gnomish bard). But what other mod may you just change your mind about halfways through the game? And would it not be faster and safer to just do a new install (we now have efficient tools to do that, it is not like in the -bad- old days anymore)?


Edited by Roxanne, 18 December 2015 - 06:24 AM.

The Sandrah Saga

another piece of *buggy, cheesy, unbalanced junk*

 


#9 The Imp

The Imp

    Not good, see EVIL is better. You'll LIVE.

  • Member
  • 5015 posts

Posted 18 December 2015 - 07:47 AM

because you may as well use shadow/ee keeper or NI to change your NPCs to whatever
Yeah, let's see, let's take Minsc(an average NPC) as the example, 5 files in SoA alone, we have 50 BG1 NPCs, and 150 BG2 NPCs. That could be as many as 1000 files you need to change, if you want the change be permanent. I would love to see you do it by hand. Well, no.
Another way to look at this: could WeiDU be improved such that it can selectively extract what it needs from the biffs instead of undoing the entire generalized biffing?
Well no, not really. We can extract files from a .bif easily, but doing that won't remove the need to keep it's presence in the game directory without uninstalling the mod that made it. So you might as well just normally install the mod as that will do the exact same thing, if it's programmed correctly. So no IF FILE_EXISTS, but instead IF FILE_EXISTS_IN_GAME or what ever it was... and then rebif the files.
Yes, the game can have the same file referred in 25 different .bif files and it will know which is the latest. Of course those 24 other files will make the reading some of the 20000 other files relatively a bit slower as it's in 24 other spaces. And it will take 25 times the files space on the hard drive. Don't mistake that to be little stuff, the original ToB game took 2.6 GB's when extracted from 5 CDs, that actually contained just 2.1 GBs actual data. Now, consider the same rate to a game that takes about 30GB's.
Lollorian could probably tell you the amount of space he saved for himself when he made the mods not biff. It was a lots.
And would it not be faster and safer to just do a new install (we now have efficient tools to do that, it is not like in the -bad- old days anymore)?
You somehow lack the basic understanding that if you can't actually install the component to the game in the first place, a reinstall will not fix the problem.

Edited by The Imp, 18 December 2015 - 07:50 AM.

Yep, Jarno Mikkola. my Mega Mod FAQ. Use of the BWS, and how to use it(scroll down that post a bit). 
OK, desert dweller, welcome to the sanity, you are free to search for the limit, it's out there, we drew it in the sand. Ouh, actually it was still snow then.. but anyways.


#10 Lollorian

Lollorian

    smiley addict

  • Member
  • 4150 posts

Posted 18 December 2015 - 09:12 AM

Relevant thread for nostalgia and teh feels :hug: :ermm:


"I am the smiley addict, yellow and round, this is my grin :D when I'm usually around :P.
When there's trouble brewing, see me post, cuz it's usually a wall o' yellow and your eyes are toast!!!"

BWP GUIDE - BWP FIXES - impFAQ - NPC LIST - KIT LIST - AREA LIST

GitHub Links : BWP Fixpack | Lolfixer | BWP Trimpack | RezMod


#11 agb1

agb1
  • Member
  • 1623 posts

Posted 18 December 2015 - 09:20 AM

Should BWS be using the Borkpack from that thread instead of (or in addition to) the Trimpack?


BiG World Fixpack (community collection of mod fixes and compatibility patches, with user-friendly cross-platform script)

 

BiG World Setup (tool to automate best-practice installation of Infinity Engine mods on Windows, with conflict analysis)

Latest version:    https://bitbucket.or.../get/master.zip


#12 Lollorian

Lollorian

    smiley addict

  • Member
  • 4150 posts

Posted 18 December 2015 - 09:33 AM

Absolutely not :)


"I am the smiley addict, yellow and round, this is my grin :D when I'm usually around :P.
When there's trouble brewing, see me post, cuz it's usually a wall o' yellow and your eyes are toast!!!"

BWP GUIDE - BWP FIXES - impFAQ - NPC LIST - KIT LIST - AREA LIST

GitHub Links : BWP Fixpack | Lolfixer | BWP Trimpack | RezMod


#13 The Imp

The Imp

    Not good, see EVIL is better. You'll LIVE.

  • Member
  • 5015 posts

Posted 18 December 2015 - 10:06 AM

The origin of the Borkpack was the trimpack, the yellow smiley monster renamed it as borkpack as it might have borked hard, but it got sorted and then went back to trimpack and then it got trimmed all on it's own into the Fixpack. All hail the yellow smiley monster !


Yep, Jarno Mikkola. my Mega Mod FAQ. Use of the BWS, and how to use it(scroll down that post a bit). 
OK, desert dweller, welcome to the sanity, you are free to search for the limit, it's out there, we drew it in the sand. Ouh, actually it was still snow then.. but anyways.






Also tagged with one or more of these keywords: bif, bws, megamod