18 replies to this topic

[Ascension64]

Working with mega-modification installations
A problem solving approach

Introduction: The Main Issue
I would like to make lucid the bony edge that modders and players have come to bicker about by starting a new topic focussed on one all-encompassing issue:

How can players and modders work together to make a mega-modification install satisfy both parties' wants?

I clarify 'player' and 'modder' as abstract terms referring to 'a person who installs and plays modifications' and 'a person who creates modifications and makes them available to the public community'. One can be both a 'player' and a 'modder'.
I clarify 'work together' as meaning both parties put in substantial effort to satisfy each others' wants.

The most important thing for me to summarise from all the discussion about the technicalities and practicalities of mega-modification installations so far are the goals of the player and the modder in relation to modifications. Please note that goals are the endpoint, the ultimate aims if you like. This does not include the methods utilised to achieve that goal. You will see what I mean if you continue reading. Also, while not all players and modders may want all that I have listed, this problem solving session is an all-inclusive attempt to satisfy all players and modders.

Goals
Players

• P1-Can use a wide range of modifications - a mega-modification installation ideally installs all modifications
  
• P2-Efficiently download modifications without tedious searching - saves time for the player
  
• P3-Simple installation process - installation with as little user input as possible, saving time and frustration
  
• P4-Simple and smart determination of installation order and compatibility - a high fidelity approach that saves time and frustration
  
• P5-A game that does not break - again, saves time and frustration

Modders

• M1-Simple maintenance and updating of authored modifications - a measure of controlled distribution
  
• M2-Simple provision of support - to reduce time debugging and increase time modding
  
• M3-A measure of respect for their work - a feeling of satisfaction

While the goals are fairly simple to set out, the means to achieve them is the main subject of the problem solving approach. I attempt to summarise the current utilised and suggested methods to achieve the goals I have listed above. It is a little messy because methods overlap in the goals they achieve in a number of cases.

Current methods to achieve goals
P1-Can use a wide range of modifications

• There are loads of modifications out there that a player might want to use, and modders make them!
  
• 'Can use' is a little more tenuous, and is the crux of compatibility; falls on the modder's sphere - Problem: Modders cannot always ensure complete compatibility

P2-Efficiently download modifications without tedious searching

• A central 'mod list' that links to the modifications desired, containing direct links to download the modifications
  
• Upload mods to an independent mirror, which may be mega-modification-specific, and directly link to the modifications - Problem: to satisfy all the goals of the modder, the host of the mirror should seek permission, but author may not reply, due to spam or loss of interest or bad contact address, or decline permission, and multiple authors may complicate permission-giving (one solution that has occurred is to set up 'unauthorised mirrors' without seeking permission)
  
• Additionally handle links by automated script - Problem: links may die at any time, such as slightly changed link due to changed version number

P3-Simple installation process
P4-Simple and smart determination of installation order and compatibility

• A sophisticated, automated installation script, of which several are around at the moment - Problem: lots of work, difficult to maintain, continuous updates may be required
  
• Use only specific versions of modifications, which may include older ones (gets around above problem also) - Problem: difficult to support by modders (M2)
  
• Automated installation script is customisable but restrictive based on order and compatibility - Problem: lots of work

P5-A game that does not break

• Fix bugs - Problem: hard to diagnose bugs with big installs
  
• Use fewer mods - Problem: contrary to P1

M1-Simple maintenance and updating of authored modifications

• Author controls distribution and/or allows, is aware, and is active with other distributions - Problem: official download mirrors may not be friendly to automated scripts
  
• Version of modification used is the most up-to-date - Problem: mega-modification installation requires continuous updating

M2-Simple provision of support

• Version of modification used is the most up-to-date - Problem: mega-modification installation bug spectrum continually changes, new incompatible components, changes to language index and component index, must scrap bug-fixing and start over

M3-A measure of respect for their work

• Ask for permission before setting up alternative distributions - Problem: author may not reply, due to spam or loss of interest or bad contact address, or decline permission, and multiple authors may complicate permission-giving
  
• Author allows, is aware, and is active with other distributions not directly controlled by him or her

Proposed methods to achieve goals
As you can already see, this is a massively complicated problem, and hopefully I have portrayed naked the goals and methods that constitute the main issue. What follows is my suggestions on the methods to satisfy the goals of the player and the modder as fully as possible. It involves work on both the players' and modders' side.

1. Using a Wide Range of Modifications
1.1. Players have access to a wide range of modifications. However, not all will work together flawlessly. This is the crux of compatibility.
1.2. Modders cannot always guarantee compatibility. An incompatibility is a conflict between components regardless of which order they are installed relative to each other. Hence, a conflict between components that occurs only in a specific order should be dealt with by adjusting installation order of the mods. If the adjusted order causes further incompatibilities with other mods, it can be referred to as an incompatibility. Incompatibilities may be unresolvable of resolvable.
1.2.1. Unresolvable component conflicts should be noted clearly. Automated installation scripts should restrict installation of two or more components containing unresolvable conflicts.
1.2.2. Resolvable conflicts may exist due to destructive changes to game resources. These can either be no longer maintained by the author, or is maintained by the author.
1.2.2.1. For destructive changes in components that are no longer maintained by the author, current modders should update these 'legacy' components. In the interim, or if any difficulties are associated with updating legacy components (including lack of permission to no one wants to update it), two situations arise.
1.2.2.1.1. If the component is based predominantly on overwriting resources, it should not be used in a mega-modification installation until the component is updated.
1.2.2.1.2. If the component is not based predominantly on overwriting resources, a patch can be created by either a player or a modder that is based on WeiDU, not replacement game resources, to address the issue, if practical. Otherwise, the component should not be used in a mega-modification installation unless it is updated.
1.2.2.2. For destructive changes in components that are maintained by the author, the issue should be flagged to the modder for prompt correction. Modders should help other modders improve their modding skills to improve compatibility, so that future resolvable incompatibilities can be avoided. In the interim, if practical, a patch should be endorsed by the modder, and created or supervised by the modder to address the issue. The patch should be based upon WeiDU, not replacement game resources, to address the issue. If the modder has elected to use only the latest version of his or her mod in the mega-modification installation, then it is his or her responsibility to notify of the update at the centralised mega-modification installation site as soon as the updated version of the modification is available. Otherwise, the older version may continue to be used with the patch.
1.3. Modders can elect whether they will support the use of their modifications in the mega-modification installation. If so, all the guidelines in this document apply. If not, then under no circumstances can the modification be used in a mega-modification installation unless the modder's stance changes out of his or her free will.
1.3.1. Should players use modifications where modders have elected not to use in a mega-modification installation, then no support shall be given by modders or players for their entire installation, regardless of whether a bug is due to the use of that modification or not.
1.3.2. If modders wish to change their stance on the use of their modifications in the mega-modification installation, they must explicitly state their new stance at the centralised mega-modification installation site.

2. Efficiently downloading modifications
2.1. To simplify maintenance and updating of authored modifications for modders, and to show a measure of respect for the work of modders, modifications should only be made available in versions and at locations at the behest of the modder.
2.2. Links to downloading modifications in a mega-modification installation should be at a centralised site that receives frequent input from modders so that updates can be made swiftly. This can be open source, allowing modders to directly modify their
2.3. The location of a modification should be determined by the modder. It can be unrestricted, restricted to 'official' sites only (SHS, PPG, G3, TBG, CoM, RPGD, Sorcerors Palace, BWL), or restricted to 'official' sites plus the centralised mega-modification installation site.
2.3.1. If the location of the modification is unrestricted, a copy of the modification should be placed at the centralised site. This ensures that the availability of the modification is stable, and simplifies handling of links.
2.3.2. If the location of the modification is restricted to 'official' sites only, the centralised site should link to the official site. Specifically, the link should point as directly to the modification as possible to avoid further clicking. Modders and site hosts should attempt to the best of their ability to ensure the file name for the modification is changed as little as possible. Ideally, a referral link, such as that used by the SHS, may be useful. Alternatively, the file name never changes between versions.
2.3.3. If the location of the modification is restricted to 'official' sites plus the centralised mega-modification installation site, a copy of the modification should be placed at the centralised site. If the modder has elected to use only the latest version of his or her mod in the mega-modification installation, then it is his or her responsibility to notify of the update at the centralised site as soon as the updated version of the modification is available. Otherwise, the older version can remain at the centralised site.
2.4. The version of the modification for use in the mega-modification installation should be determined by the modder to ensure that modders are comfortable with the provision of support for their modification. The modder should nominate if they would like to use only the latest version of the modification, or allow older versions of the modification to be used.
2.4.1. If the modders would like to use only the latest version of the modification in the mega-modification install, the mega-modification installation should be updated as soon as possible after the version of the modification is updated. It is the modder's responsibility to ensure that the links from the centralised site are updated promptly. If the modder places no restriction of the distribution of the modification, an updated copy of the modification should be placed at the centralised site as soon as possible, preferably under the responsibility of the modder. If the modder allows distribution of the modification to the centralised site, it is the modder's responsibility to update the modification version at this site as well.
2.4.2. If the modders allow older versions of the modification to be used in a mega-modification installation, the modder should elect a particular version and the distribution method of the modification as in Section 2.3. The most direct link to the file should be placed at the centralised mega-modification installation site.
2.4.3. If the modder is indifferent to the older version of the modification to be used in a mega-modification installation, the current version shall be assumed.
2.4.4. So long as the modder does not wish to change the version of the modification he or she would like to be used in a mega-modification installation, that version will always be used regardless of updates to the version of the modification.
2.4.5. If the modder wishes to change which versions of the modification he or she would like to be used in a mega-modification installation, the guidelines should be followed as per using the latest version of the modification outlined in Section 2.4.1, but using the particular version of the modification that he or she elects.
2.4.6. If the modder elects an older version, he or she must provide support for that version of the modification.
2.5. If the modder does not elect a location and version of the modification to be used in the mega-modification installation, every attempt should be made by players and modders alike to contact the modder and receive an election. Several situations arise.
2.5.1. If the modder responds and elects a location and version, then the procedures should be followed as described above.
2.5.2. If the modder responds and shows indifference, then the distribution of the modification can be deemed unrestricted, and the versions allowed are deemed to be any.
2.5.3. If the modder responds and declines use of the modification in a mega-modification installation, then Section 1.3 is followed.
2.5.4. If the modder does not respond after every attempt at communication is made for a consistent 3 months, then the modder's response is deemed indifferent, and Section 2.5.2 is followed.
2.6. If a modder wishes to change their stance on any topic regarding the distribution or version of the modifications for use in the mega-modification installation, they must explicitly state their new stance at the centralised site.

3. Installing modifications
3.1. Installing modifications covers the determination of modification compatibility in regards to installation order and installation viability, the miscellaneous determination of installation order, and the actual installation process itself
3.2. All known modification compatibility issues must be taken into account in the determination of installation order and installation viability.
3.2.1. It is the responsibility of the modder to be aware of all known compatibility issues between their modifications and other modifications. This allows the modder to advise properly of corrections to installation order, create or supervise patches, and make corrections to their modifications if required. The author of a modification should have the most informed say on installation order and viability for their modification, and thus should decide where the modification should go relative to others in a mega-modification installation. The modder must have a valid and confirmed reason for placing a modification at an absolute or relative position in the installation order. Unfounded, whimsical, and dubious reasons should not be considered.
3.3. Installation order should be handled at the centralised mega-modification installation site, preferably in an open-source manner or by a custodian that follows the above guidelines.
3.3.1. When determining installation order for the current usable versions of modifications, modifications should be placed in the following order: (1) identify modification groups of the smallest size possible from all modification with known compatibility issues (these groups contain all modifications where there is a compatibility issue between any two members) and assign relative order within groups, (2) put all modification groups in an absolute order from the group containing the largest number of members to the group containing the smallest number of members, (3) 'miscellaneous' modification with no known compatibility issues shall be placed in alphabetical order after all modification groups. Prepositions in modification names will be considered when determining this order.
3.3.2. If two components are not viable together under any circumstances, an either-or approach to the installation should be used, but the determined installation order should not be changed, even if a compatibility issue is alleviated upon withdrawal of a component.
3.3.3. Every version of the installation order should be tagged with a code, preferably the date rather than the version, and stored in a repository, to enable perusal when dealing with bugs.
3.4. The installation process itself should comprise of a batch method that takes into account the currently determined installation order.
3.4.1. Each modification should have an accompanying readme that can be perused as the player chooses which modifications that he or she would like to install. Readme files should be mirrored at the centralised mega-modification installation site. As for modifications described in Section 2, it is the modder's responsibility to ensure that these readme files reflect the version of the modification used in the mega-modification installation.
3.4.1. The choice of components to install should be customisable to the player and include restrictions on components that are not viable together and components that require other components.
3.4.2. From the nominated list of components, an automated script should generate a batch file that handles the installation process with as little input from the player as possible. The automated script should also ensure that the installation will occur with the least probability of error. This includes validation of the Baldur's Gate II installation. Throne of Bhaal version 26498 is assumed.

4. Dealing with bugs
4.1. While there are numerous compatibility issues and bugs known, there are just as many, if not more, that are not yet known.
4.2. All bugs encountered during the playing of a mega-modification installation must be reported at the centralised mega-modification installation site, or a forum thereof.
4.2.1. Bugs should be tracked in a database, preferably open-source, which contains the details of the bug and the status on its verification, the diagnosis of the causative modifications, and fixed state.
4.2.2. Every single bug should be reported separately, regardless of whether they were encountered in quick succession, or in the same game. A single bug is a bug that deals with a single situation in the game. Some examples include an incorrect dialogue, a game crash, an unending cutscene, and a missing item icon. As an additional example, a creature with an incorrect dialogue and a bad script should be considered two bugs and thus reported separately.
4.2.3. Players should make every attempt to determine whether their bug has already been reported. If it is, they should confirm the bug and give any further information, if available.
4.2.4. Players with sound modding knowledge are in the best position to perform bug diagnosis. A guide should be present to provide details as to the tools available for the diagnosis of bugs and how to use them. This will encourage more players to diagnose bugs and improve confidence paying a mega-modification installation.
4.2.5. Single bug reports must each contain: (1) the user reporting the bug, for correspondence if further information is required, (2) the WeiDU.log of the game containing the bug, (3) the code of the installation order used, (4) date and time of the bug report, (5) the area number, coordinates, creature name, item name, spell name, situation, cutscene, NPC, or similar thereof associated with the bug, (6) a succinct description of the bug.
4.2.6. Users are advised to keep a separate saved game of the bug to assist with bug diagnosis.
4.2.7. Players should be instructed with the use of the CLUAConsole and Debug Mode with regards to reporting bugs so that the most complete information can be garnered in a bug report.
4.3. Diagnosed bugs can be of two types: single modification bugs, or compatibility conflicts
4.3.1. Bugs that arise from a single modification are due to a single modification itself and not with a mega-modification installation. These bugs should be separated from bugs affecting the mega-modification installation (i.e. compatibility conflicts) and forwarded to the corresponding authors of the modification containing the bug.
4.3. If the bug arises from a compatibility conflict, these bugs should be separated from bugs that arise from a single modification. Steps should be taken as per Section 1.2 to resolve the conflict, if possible. If the issue is solved partially or fully by adjusting installation order, the necessary steps should be taken as per Section 3.3.
4.4. If the bugs are not game-breaking, for simplicity, it is strongly advised that players continue to play without re-installation, installation of additional components, or even the correction of the bug. In other words, players should 'live' with these bugs. As annoying as a bug may be, a re-installation is even more annoying, and making any raw changes to the game will complicate bug tracking later on. This should apply to the smallest bug, such as an item incorrectly flagged undroppable.
4.5. If the bugs are game-breaking, a workaround should be utilised as quickly as possible to allow the player to continue enjoying the game. The workaround should most closely resemble the natural flow of the game and avoid touching on aspects of the game astride from the game's natural flow. Ideally, this would represent the smallest change possible to the offending resource to correct the bug.
4.5.1. Workarounds for game-breaking bugs should be stored with the installation order it is associated with and should be the same for all players of that installation order. Workarounds should be hosted at the centralised mega-modification installation site, and be associated with the bug report it temporarily corrects.
4.5.2. Workarounds should be inherited into updated installation orders so long as the version of the modifications affected by the workaround do not change. Otherwise, workarounds should not be inherited unless the same bug is reported in the updated installation and the affected resources have not changed at all. If the affected resources have changed, the workaround must be updated to reflect those changes.

Pretty lengthy, but necessary given the complicated nature of mega-modification installations. I will try to summarise the duties of the modder and the player below.

Duties
Modder

• Nominate and notify of all changes of stance whether their modifications can be used in a mega-modification installation (yes, no). If so, nominate how it will be distributed (unrestricted, restricted to 'official' sites, restricted to 'official' sites plus centralised site) and which versions can be used (latest only, specific version, any version). If latest version is nominated, notify the centralised site of all updates to modifications and links. If older versions are nominated, the old version needs to be made available at the distribution method nominated by him or her.
  
• Endorse, and create or supervise patches and workarounds as necessary.
  
• Improve modding skills, help others improve modding skills, and update legacy code to modern standards, to emphasise compatibility.
  
• Be aware of the known compatibility issues between their modifications and other modifications.

Player

• Develop skills in use of the console, use of debug mode, and bug diagnosis.
  
• Report bugs correctly at the centralised site.
  
• Stick strictly to the modifications mentioned in the installation order and the installation order itself, or otherwise abstain from seeking any help.

Both

• Set up and maintain the centralised mega-modification installation site, including links, readme files, script generation, bug report database, installation repositories, and bug notification for bugs arising from single modifications.
  
• Communicate with modders who have not nominated for their modifications to be used in a mega-modification installation to get a response.
  
• Communicate with modders who have not notified of distribution and version of their modifications to be used in a mega-modification installation to get a response.
  
• Determine installation order and installation viability rules governing the automated script.

The only problem with all of this is hard work and time, but if no one is willing to put in the hard yards to make things work for the better of both modders and players, the mega-modification installation should not work at all.

Discuss.

Edited by Ascension64, 29 December 2007 - 10:55 PM.

[bigmoshi]

Hi Ascension64,

Firstly, bravo. Its certainly a well-thought & articulated post. I can't imagine how you fast you work - modding & such things.

I like things simple, so from my perspective there really only two issues to consider.
One, the technical side. And two, the management side.

On the technical side, couple of issues to consider:

• Setup of a system with mod-version control & bug tracking, auto-installation & everything else - simple
  
• Component-conflicts & versioning-differences for ongoing projects - create a stablised core, best if it includes NEJ and build on from there. Else if it takes 2-3years to stablise, we'll be playing with 2006 mod versions in 2009. Customisation can start from that core, rather than from the base SoA/ToB installations.
  
• Considering the number of skilled & dedicated programmers in the community like thebigg, sconrad & so many more, this will ultimately be resolved. The issue is when? 2 years? or 5 years?

The bottom-up approach has been tried maybe too many times. Honestly, I find that for a mega-mod to be really successful and come up to speed, perhaps it's time to try a top-down approach. Basically:

• Maybe the 4-5 large mod development teams (eg. SHS, G3, PPG, wyrmlair, Mystra) could consider forming a committee to speed up the process. It can't be a SHS mega or G3 mega etc, its a BG mega.
  
• Collectively endorse a project and spell out the direction. That voice will be heard by all modders and players alike. I'm not sure, perhaps this has been done before, perhaps not.
  
• Market that idea not only to the modding community but to the BG community at large. There are probably a hundred times more players than modders.

I like the way berelinde spelled it out as Can't we all just get along? (sorry, slightly out of context). We need a bus driver to pick up all the passengers (modders/players/etc) and make sure most of us are moving in one direction.

Going back into my game bb

[Ascension64]

Component-conflicts & versioning-differences for ongoing projects - create a stablised core, best if it includes NEJ and build on from there. Else if it takes 2-3years to stablise, we'll be playing with 2006 mod versions in 2009. Customisation can start from that core, rather than from the base SoA/ToB installations.

Intuitive enough. After all, a mega-modification installation is defined as a modded ToB with at least two of <that long list of acronyms, see here>. If we start now, all the current versions of those mods can be considered core. Please remember though that some people probably use a less-than-core (probably TDD being the most frequently omitted mod, no offense meant), so any immediate fixes should not automatically be installed without first detecting what is installed. This is easily done if a WeiDU patch addresses fixes (a bit like S&H of the old BP-BGT-NeJ mod) rather than destructive overwriting.

I think I also mentioned this in another thread, but you probably won't find compatibility issues with a core. It is much more common for the smaller mods on top to cause turmoil.

Considering the number of skilled & dedicated programmers in the community like thebigg, sconrad & so many more, this will ultimately be resolved. The issue is when? 2 years? or 5 years?

Faster if we worked on one project. Slower if twenty people worked on twenty projects, and that's the problem. We currently have three auto-installers going, and that isn't going to help modders, players, or progress.

Maybe the 4-5 large mod development teams (eg. SHS, G3, PPG, wyrmlair, Mystra) could consider forming a committee to speed up the process. It can't be a SHS mega or G3 mega etc, its a BG mega.

I'm struggilng to work out what we'd do as a 'mega-council'. We are all responsible for our own mods. Everyone teaches each other compatibility-friendly modding skills, time willing, and some take it on board, and some don't, as you would naturally expect. If you think we can query all modders on our site for details on mega-mods then we could do that, but the sites can do that without idly bantering with each other.

Collectively endorse a project and spell out the direction. That voice will be heard by all modders and players alike. I'm not sure, perhaps this has been done before, perhaps not.

Well, I'll wait for more comments on this thread, but I presume that it's currently too lengthy and legislation-like to make for a decent one-session read. I am open to revisions and alternatives to my take, but we need discussion before we can agree on anything.

Market that idea not only to the modding community but to the BG community at large. There are probably a hundred times more players than modders.

As in market a mega-mod? For sure, it is possible.

I like the way berelinde spelled it out as Can't we all just get along? (sorry, slightly out of context). We need a bus driver to pick up all the passengers (modders/players/etc) and make sure most of us are moving in one direction.

Makes me think of Waukeen, but surely the Emperor of the Modding Community I don't think will work as well as a 'ruling council' containing everybody.

[bigmoshi]

I like the way berelinde spelled it out as Can't we all just get along? (sorry, slightly out of context). We need a bus driver to pick up all the passengers (modders/players/etc) and make sure most of us are moving in one direction.

Makes me think of Waukeen, but surely the Emperor of the Modding Community I don't think will work as well as a 'ruling council' containing everybody.

>_< okay BUS COMPANY - NOT BUS DRIVER

[Kobold]

Wow this is a lot of stuff yo u wrote, but as a newbie to this whole BG Mod installing and Management thing I think you made a lot of valid points. For me as a player it takes lots of time to find out what is compatible and what is not (and the problem is I don't have enough time to really look into it, which lead to me asking stupid questions and making stupid mistakes). SO something like a linux-like packet-management system would seem great to me.

I would like to comment on some individual thoughts of your:

3.2.1 It is the responsibility of the modder to be aware of all known compatibility issues between their modifications and other modifications.

I think this is a quite hefty request on a modder and perhaps I did misunderstand the point. Actually I think a more linux-like approach could help the modders. Other modders and perhaps even experienced players could hand in patches for mod incompatibilities and it would then be the responsibility of the modder to check their feasibility or sth. like that.

Player

* Develop skills in use of the console, use of debug mode, and bug diagnosis.
* Report bugs correctly at the centralised site.
* Stick strictly to the modifications mentioned in the installation order and the installation order itself, or otherwise abstain from seeking any help.

Both

* Set up and maintain the centralised mega-modification installation site, including links, readme files, script generation, bug report database, installation repositories, and bug notification for bugs arising from single modifications.
* Communicate with modders who have not nominated for their modifications to be used in a mega-modification installation to get a response.
* Communicate with modders who have not notified of distribution and version of their modifications to be used in a mega-modification installation to get a response.
* Determine installation order and installation viability rules governing the automated script.

Actually as a player I would love to help. And now a stupid question regarding that. IS there a "Players Guide to Debugging" or sth. like that, cause as I said before the amount of time availalble to me forbids to really dig deeply into it (by trial and error or by learning the hard way).
A probably more or less unresolvable problem? is the necessarity of a install order? Ideally every mod should at a point know about all possible dependancys in the megamod, so installation order wouldn't play any role but this is really probably impossible.

Just my two cencts and thanks for all your efforts Ascension64 (and of course all modders out there). I really apprecitate it.

Kobold

P.S. And no I haven't looked into any of the automated install tools yet. (I didn't know they existed).

Edited by Kobold, 31 December 2007 - 03:53 AM.

[Ascension64]

3.2.1 It is the responsibility of the modder to be aware of all known compatibility issues between their modifications and other modifications.

I think this is a quite hefty request on a modder and perhaps I did misunderstand the point. Actually I think a more linux-like approach could help the modders. Other modders and perhaps even experienced players could hand in patches for mod incompatibilities and it would then be the responsibility of the modder to check their feasibility or sth. like that.

I had to choose my words carefully here. The key word is 'aware', meaning that the modder must know that there is a problem between mod X and mod Y. Note that I actually haven't said that the modder must make the patch. The remaining clauses in that same section state:

This allows the modder to advise properly of corrections to installation order, create or supervise patches, and make corrections to their modifications if required. The author of a modification should have the most informed say on installation order and viability for their modification, and thus should decide where the modification should go relative to others in a mega-modification installation. The modder must have a valid and confirmed reason for placing a modification at an absolute or relative position in the installation order. Unfounded, whimsical, and dubious reasons should not be considered.

Since the modder appears to the person who is the most knowledgable about what their mod changes, it is reasonable to conclude that the modder knows best how their mod works with others. Thus, when a compatibility issue arises, the modder is in the best position to know what is wrong. That is why they must be 'aware' of the compatibility issues with other mods.

If you continue reading, I say that the modder can 'create or supervise' patches. So, the modder can make the patch him or herself, or endorse and supervise a patch made by another modder who is not a co-author of the modification in question or by players. In these cases, the patch needs to be endorsed and supervised because, for quality control, the modder knows best if the patch will work flawlessly, or will impinge on other aspects of the mod.

So, the whole idea of getting the modder involved is because he or she has the knowledge of what their modification does. Note that they might have not ideas about how their patch might affect other modifications, but niether do the players themselves. Hence, a collaborative approach between other modders become important in ensuring that a patch doesn't break something else, which would greatly complicate bug diagnosis and resolution later on.

Actually as a player I would love to help. And now a stupid question regarding that. IS there a "Players Guide to Debugging" or sth. like that, cause as I said before the amount of time availalble to me forbids to really dig deeply into it (by trial and error or by learning the hard way).

I am currently in the process of writing one. It will cover CLUAConsole, Debug Mode, NearInfinity, DLTCEP, and WeiDU. Don't know when it will be ready, though.

A probably more or less unresolvable problem? is the necessarity of a install order? Ideally every mod should at a point know about all possible dependancys in the megamod, so installation order wouldn't play any role but this is really probably impossible.

I got caught on the word 'ideal'. Modding is so stochastic and variable, as is the nature of entropy. So now, we need to put order into that chaos, and while modders are continuing to make things more compatible and 'installation order-friendly', that is going to take quite a lot of time, and installation order needs to be considered in the interim.

[bigmoshi]

Btw, there's a section 3 on Generic XML Builder for custom mega-mods in the SMM thread, in case the title din't reflect it. http://www.shsforums...showtopic=30695. Quite a simple system, just uses a matrix map to resolve conflicts. Language & templating features are listed there as well. Any other ideas?

To Ascension: I'm working on a version-independent patch script, but I'm not sure whether my understanding of weidu installations is enough. Can I pm you later regarding some issues on weidu?

[Ascension64]

Btw, there's a section 3 on Generic XML Builder for custom mega-mods in the SMM thread, in case the title din't reflect it. http://www.shsforums...showtopic=30695. Quite a simple system, just uses a matrix map to resolve conflicts. Language & templating features are listed there as well. Any other ideas?

Exactly how it works is slightly complex since I am not too familiar with the coding you have used (not a XML, .NET person). Does the matrix map simply work exclusively (if x.y has a value of 1, it means you cannot install both x and y at the same time)? The way you set it up looks quite similar to how kovarex's installation works (I see he is currently modifying his at the moment). I suppose later you could add a less black-and-white method that could involve more values for fields that could define further rules in installation, such as order, and display of custom text explaining what's wrong with using the two mods, or flag compatibility patches. Perhaps something for later after the basic system is set up.

The amount of time that you have spent setting this up for users is greatly appreciated. That also goes for everyone else trying to make mega-mod installs easier.

To Ascension: I'm working on a version-independent patch script, but I'm not sure whether my understanding of weidu installations is enough. Can I pm you later regarding some issues on weidu?

Sure.

Edited by Ascension64, 02 January 2008 - 05:32 PM.

[bigmoshi]

Btw, there's a section 3 on Generic XML Builder for custom mega-mods in the SMM thread, in case the title din't reflect it. http://www.shsforums...showtopic=30695. Quite a simple system, just uses a matrix map to resolve conflicts. Language & templating features are listed there as well. Any other ideas?

Exactly how it works is slightly complex since I am not too familiar with the coding you have used (not a XML, .NET person). Does the matrix map simply work exclusively (if x.y has a value of 1, it means you cannot install both x and y at the same time)?

I mainly used PHP & MySql database to handle the online configuration editor. I'm not exactly sure whats the best way to handle custom mega-mods yet, so I made it flexible enough to handle uni-directional associations, ie x -> y but not y -> x. Thus, it should handle mutual exclusivity if the values are set twice, ie bi-directional association. I'm more of a visual person, so here's it works.

Blue means same values across the Black axis. Red means different values from the corresponding blue. (Doesn't mean all the blues == same value though)

The way you set it up looks quite similar to how kovarex's installation works (I see he is currently modifying his at the moment). I suppose later you could add a less black-and-white method that could involve more values for fields that could define further rules in installation, such as order, and display of custom text explaining what's wrong with using the two mods, or flag compatibility patches. Perhaps something for later after the basic system is set up.

That is possible, probably need to extend another table in the dbase for that extra conflict info. Anyway, I got really lazy to fill up the table, so I just parsed the strings from weidu log and whatever I could find in my xml file using Excel. Eg. from ~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #0 // BG2 Fixpack - Core Fixes, using "=RIGHT(L3, LEN(L3)-(FIND("// ",L3)+2))" to get "BG2 Fixpack - Core Fixes". So whatever is needed should be filled up in excel and imported into the dbase.

The amount of time that you have spent setting this up for users is greatly appreciated. That also goes for everyone else trying to make mega-mod installs easier.

Frankly not even a fraction of the time you guys spent on the mods.

To Ascension: I'm working on a version-independent patch script, but I'm not sure whether my understanding of weidu installations is enough. Can I pm you later regarding some issues on weidu?

Sure.

Ok, I'll disturb you later. Thanks.

[bigmoshi]

Maybe this would be easier to digest. Extracted from http://www.shsforums...p...0560&st=60#

Do spend some time reading his ideas & comment on it for the good of mega-modding.

A body/committee of reputation should take charge of the process, such that modders would be assured and confident of the quality control of that process. That way, things can proceed with less hiccups & arguments.

A body would need to consist of both modders and players for there to be any evenness in discussion and proceedings. I think we'll find one as we need one.

Avoid the issue of rights entirely by maintaining an "out-by-default" policy where the selection process for mods should be "out" by default - "in" only when agreed upon by the owner/developer/whoever.

This ensures that we don't step on anyone's toes accidentally. I wish to add an exception that an 'out-by-default' where the author is no longer interested or cannot be contacted to the best of the ability of modders and players alike for 3 months becomes 'in'. Otherwise, you will never get some of the older mods being in an mega-modification installation. I described this in the problem solving approach method.

By opting "in" to want your mod included in the mega-mod, the modder has to or would implicitly agree that it will be distributed as such and such.

Hmm, sounds like a compromise. Already, a number of people wish for their mods to be used in a mega-mod (and why not, more players to boot up their mod!) but want to be more able to support and maintain it better. This is why I propose that the mod authors elect their distribution method and version. I do not see any added difficulty with maintaining links to all over the place. The tricky part is to ensure that the links work and any updates are made ASAP. I see it as the modder's responsibility to advice explicitly in a mega-mod area that a mod has been updated, and they want the mega-mod to use the new version of the mod. If older versions are alowed, this make it simple, so long as the older version used is available and the mod author doesn't change his or her stance on the version to use often.

Start adding the larger fruits to the basket before adding the smaller ones - same way you try to fill a container with rocks, stone, gravel then sand.

Plausible. I don't see this making any difference to the way it will evolve compared to picking up random flotsam and jetsam and throwing it all together. This is because mega-mod installation order is determined by compatibility, not size. However, a mega-mod install might develop fast if you went from author most available to author least available.

Edited by bigmoshi, 08 January 2008 - 07:48 AM.

[bigmoshi]

Stupid Question -->

Spoiler

Any chance that this will be happening?

[Ascension64]

Stupid Question -->

Spoiler

Any chance that this will be happening?

Not stupid at all. Two people have commented so far. No other modder has endorsed/refuted the idea. The thing is, this is no an iron fist rule, and really needs a consensus to go forth. So, if people are interested but are simply going to be passive, nothing is going to happen.

[cmorgan]

I support Ascension64's idea. I will assist in what way I can, at the edges, as long as authorship is respected and people are working in good faith to allow more people's stories to be told.

Unfortunately, my assistance is of limited merit, as I play the exact opposite of Megas - small, targeted installs on EasyTutu_ToB and then small, targeted BG2 installs with extremely limited numbers of mods.

What grunt recode, research, or information gleaned from current practices I can provide, I will lend.

[Nimloth05]

I don't know if this will help you - I have very little knowledge of all this modding stuff but I have some experience in deploying complex software. So here are my very quick thoughts:
A mod needs something like a manifest/XML which describe deployment details for a particular mod. This file gives an installer all information it needs to know to correct install this mod.

In such a file the modder can say:

- which mods must be installed BEFORE.
- which mods and there version it depends on - I don't know if this is necessary but it would help build core mods and fixes which every other mod can use.
- If an update destroys a running game.
- List of mods which are not compatible to this mod.

Which this information one could write an installer who reads out all the XMLs and create proper installation order. I imagine something like this: The installer shows a list of all mods on a mod server. The user picks some and the installer create the correct order, download and install them. But this would require, that every mod has such a manifest.

So instead of one install XML Script like the one from SMM you have an XML for every mod which can be assembled to one installation.

Hey guys, I hope I could help you at least a little. All the mods you have done and all the effort - it's just amazing.

Edited by Nimloth05, 02 March 2008 - 03:46 AM.

[Ascension64]

They are some useful features you mention there to help determine installation order. For mods with unknown compatibility to the mod in question, can the XML script put them alphabetically? Can it also order installations in blocks starting with conflict blocks (a group of mods with some compatibility rules applied) and then the final non-conflict block (the mods with no known, or no incompatibility data)? Can the XML script differentiate between different versions of the same mod, even if the name of the mod is not changed? Can components of mods be considered rather than entire mods themselves (including ordering of components in a mod as a block)?

Well, I think the XML for each mod could be grouped into one larger manifest that people can download. That would save some effort on behalf of the modders with re-distributing their mods just because they might have screwed up the XML. Instead, they could just post their XMLs to be packaged into a master XML archive.

[Nimloth05]

For mods with unknown compatibility to the mod in question, can the XML script put them alphabetically?

Yes, this would be possible - I first assumed a "undefined order" for mods with nod order requirements but alphabetically would be as possible as well.

Can it also order installations in blocks starting with conflict blocks (a group of mods with some compatibility rules applied) and then the final non-conflict block (the mods with no known, or no incompatibility data)? Can the XML script differentiate between different versions of the same mod, even if the name of the mod is not changed?

Hm, it depends: A mod can for example say: "attention, i'm imcompatible with mod A in Version 1.0, I need Version 1.1". So the installer could prevent to install wrong version of a mod...

I'm glad my suggestion if of some use. I can make an example XML for better discussion.

Can components of mods be considered rather than entire mods themselves (including ordering of components in a mod as a block)?

Hmm... stupid question: Why is this feature necessary? I don't know the mod world of BG so...

Well, I think the XML for each mod could be grouped into one larger manifest that people can download. That would save some effort on behalf of the modders with re-distributing their mods just because they might have screwed up the XML. Instead, they could just post their XMLs to be packaged into a master XML archive.

I will make an example XML in a few hours and then we see thins more clearly... (-:

[10th]

Can components of mods be considered rather than entire mods themselves (including ordering of components in a mod as a block)?

Hmm... stupid question: Why is this feature necessary? I don't know the mod world of BG so...

It's necessary because mods in BG/IE Games can be installed piece by piece and sometimes it's only one/several component/s of a mod which is/are incompatible to others while the rest of the mod has no compatibility issues of any kind.

10th

[Nimloth05]

Mod to install: A
Other Mods: B, C, D

manifest:

<mod>
<identification modId="A" version="1.2.3" />
<description>
This is a very cool BG mod.
</description>
<dependency>
<mod modId="B" version="1.0.0" />
</dependency>
<installOrder>
<before modId="C" />
<after modId="B" />
</installOrder>
<incompatible>
<mod modId="B" version="< 1.0.0" />
<mod modId="D" version="undefined" />
</incompatible>
<updateDestroysRunningGame />
</mod>

identification:
----------------------
modId: The modId is a mod unique identifier. Every mod which should be installed with this system needs such an id.
version: version string. Undefined means every version of the mod.

description:
-------------------
some text for the gui...

dependency:
-------------------
A mod can have oher mods as dependency. The mod can not be installed if the specified mod isn't present. In the install List or in the isntallation.

In this example: Mod A can not be installed if mod B is missing.

InstallOrder:
---------------------
This is very special here you can specify for which mods this mod needs to be installed before.

before: This mods needs to be installed before the specified mod.
after: This mod needs to be installed after the specified mod.

This is very simple but dangerous: You could create an unresolvable installation order.

In this example: Mod A must be installed before C but after B, so the installation order is: B, A, C

incompatible:
---------------------
This mod is incompatible with the specified mods.

In this example A can not be installed when mod B in Version < 1.0.0 is present.


updateDestroysRunningGame:
-------------------------------------
If you install an update of this mod, does it destroy a running game? In this case yes, if not you can remove the whole tag.


In my imagination every mod will be distributed as a rar archive. Every archive has it's own manifest. But I see the problem: Not every mod is a rar and what if the XML is
erroneous? For the last problem we could use a other file format. Simpler as XML:


manifest
identification: A, 1.2.3
description: This is a very cool BG mod. But now, this text may not have a line break.
dependency: B, 1.0.0";
installOrder-before: C
installOrder-after: B
incompatible: B, < 1.0.0; D, undefined
updateDestroysRunningGame=true


We could imagine that this manifest is outside of the archive so the manifest could be altered without republish the archive itself. This could be an advantage but also an disadvantage.


Ok, now the components of a mod... Do I understand this correctly when I say: "Some mods or components can not be installed if a other mod or component is installed"? So a specific mod is
incompatible with component X of a other mod... We could rearrange the manifest so that the incompatible stuff is for a specific component:

manifest
<mod>
<identification modId="A" version="1.2.3" />

<description>
This is a very cool BG mod.
</description>

<dependency>
<mod modId="B" version="1.0.0" />
</dependency>

<installOrder>
<before modId="C" />
<after modId="B" />
</installOrder>

<components>
<component id="IdOfTheComponent" label="short label for the gui">
<mod modId="B" version="< 1.0.0" />
<mod modId="D.componentIDX" version="undefined" />
</component>
</components>

<updateDestroysRunningGame />
</mod>

We append the ID of the component on the ID of the mod so we can identify them.
But hmm... I have further questions:
- Have components Versions of there own?
- Is it enough to specify an installation order for mods only or do we need component info here?
- Has every mod at least one component?

Edited by Nimloth05, 05 March 2008 - 02:09 PM.

[Ascension64]

That looks like a good building point. I wonder if bigmoshi is around to help with this. I won't make things more complicated yet (since it will get complicated).

Besides the continuing development of the script, perhaps you could code up a real example (non-functional, but conceptual). Someone needs to give us an example, though, since I'm not too familiar specific compatibility issues. Anyone out there able to give us 3+ mods with stringently defined order of installation?