70 replies to this topic

[Azkyroth]

I thought about the people notifying the author to change it. I imagine that if the author is aware that his review is generally perceived as not objective, he might change it hmself.

Again, I doubt that'd work - if somebody posts a review that is perceived enough unobjective as to have people lament about that, chances are that he's willingly being unobjective and thus won't cooperate. Even if he has no ill intent, it would simply mean that he has a peculiar opinion of the mod, and thus gives a review that isn't viewed as objective. The forum idea is good, though.
And I don't want to be nitpicky, but "this mod adds 50HP and some spells to a creature" isn't an objective review, it's a description

Perhaps if we were to stick to descriptions like berelinde suggested for the actual review, and as a sidebar or something include a poll with users' ratings of various features of the mod? Then we'd get both "what it does" and "what people generally think about it."

[Bookwyrme]

I thought reviews were, by their very nature, subjective.

The readme (usually) provides a description of what the mod does, the review provides information about what the reviewer thought about it.

They can be helpful, yes, but with mod reviews as with book or movie reviews, it's a matter of knowing how often you tend to agree with a particular reviewer. If they tend like mods (or books, or movies, or cars) for the same reasons you do, then it's a fair bet that the next mod they like will be one you also will like. If not, then--not.

A good review does, it is true, go past "This was terrible!" or "This was wonderful" to explain what or why it was terrible or wonderful, but objectivity seems to be beside the point.

[Ascension64]

I'd like to see one such review before I comment on its appropriateness and usefulness. All of this bickering on a hypothetical something that hasn't been fully fleshed out at all doesn't seem to make any progress - there aren't finite moves like a chess game.

If the pro-review people really want to push this idea forward, you'd be better off defining all the specifics (structure, format, criteria, content), even provide an example, instead of delving into vague possibilities that all the anti-review people can pull out all of Murphy's tricks on.

Then, we'd all be in a better position to look at the work and credit/discredit/flame/contribute/improve it.

No policy-maker will ever agree with you if you just said to them "I want to have this kind of policy" without telling the specifics of how you plan to implement the policy, even if it is a draft format. But you got to have something there - no egg shells please.

objective reviews of mods, one that gives general information, pros, and cons, preferably in a standard format, with open discussion to follow

i.e. This doesn't mean anything to me. You haven't defined your particular understanding of all the terms you use, and it is not specific.

Edited by Ascension64, 15 June 2007 - 08:13 PM.

[berelinde]

An example will be forthcoming. While I cannot guarantee that it will satisfy any reservations you might have regarding the usefulness of the idea, I do understand the need to clarify with example rather than endless debate. Please understand that it takes considerable time to put something like this together. My time is no more plentiful than anyone else's: 24 hours per day, no more, and like everyone else, I am forced to divide it between working, commuting, family responsibilities, and prior commitments. I had penciled this project in for Thursday evening, but several more pressing matters forced me to postpone it until Sunday morning.

[Chevalier]

I can write unbias (not part of the development team and don't know any of them) review of how I like TDD ( I know I don't have any taste). I am sure the bigg and Kulyok would fully agree with me And I like many of Ed Greenwood's and Elaine Cunningham's FR books too!

[cmorgan]

I missed an opportunity to jump on the Domi/Ashara Fanboy Bandwagon? OK, that will teach me to ignore a thread... sorry. Move over, the bigg.

OK, reviewed whole (holy cow) thread. Thank goodness no one started a discussion of the Eisnerian Lens versus Objective Reality and its relationship to the concepts of both Clockwork Universe™ and Uncertainty Principal™ (though I know you really wanted to, Azkyroth ).

Toughest thing about havng so many smart people discuss things to death is that everyone appears to have a big piece of the truth, and there is no one way to be "correct". One of the strengths I find as a player is that I can read about a mod on several different forums, look at what the modder writes about versus how they deal with their public and other modders, and then look at who (what players/modders) seem to like the mod. I don't think this can be replicated by any of the systems presented here. The good news is that

it doesn't have to... it just is one more good golf-swing at giving people advanced information to make their decisions!

[stating the obvious]
I think it is a good thing that everyone is so into discussing this, because it brings up lots of frustration on the part of modders, who work hard to present their stories. A "bad" review (in form, in content, in approach, in subjectivity/objectivity) may mean less people deciding to see the story/the modders' hard work.
[/stating the obvious]

If worst came to worst, you could have a "He Said/She Said" approach, and have two side by side reviews, one positive and one negative.

I would just like to point out that this does not have to be the Holy Grail of Mod Review. Though you can call me Tim...

Edited by cmorgan, 17 June 2007 - 05:53 PM.

[Kulyok]

A "bad" review (in form, in content, in approach, in subjectivity/objectivity) may mean less people deciding to see the story/the modders' hard work.

Nah, it's all publicity.

[berelinde]

Here is one possible sample review. As I have not played any mod with the intention of reviewing it, I thought it might be better to use one I know well. While I admit that it is difficult to review my own work, I nevertheless attempted to do so.

If enough people think the idea has merit, I'd like some kind of standardized format for them, so comments on the format are appreciated as well. Naturally, quest mods would require a different kind of format, as would tactical, item, or comprehensive mods.

NPC Mod Review Template

Name: Gavin
Version reviewed: version 1.2 beta
Mod author: berelinde
Website/DL location: http://www.gibberlings3.net/gavin/

*General Information: Neutral Good cleric of Lathander, found outside in the Beregost Temple area.
Str: 16 Dex: 12 Con: 16 Int: 12 Wis: 17 Cha: 12
Stat total: 85

*Bio: Gavin was born in Ulgoth's Beard, one of the younger children of a sailmaker and a midwife. He had an unremarkable childhood, until, at the age of 12, he discovered he had a talent for healing. His father was a follower of Tyr, but his mother worshipped Lathander. They were quite surprised to find that their rambunctious son had a calling to the priesthood, but they enthusiastically supported his decision, nevertheless, accompanying him to Beregost to begin his novitiate. His service to his god and his temple was devout, but otherwise unimpressive, and he was ordained a Dawnbringer at the age of 19. Since then, he has spent most of his time performing varied services for the temple, but Gavin's advancement within the ranks has been slow. Gavin is 28 when you meet.

*Starting Equipment: splint mail, large shield, helmet, war hammer, sling, bullets, potions of healing, antidote

General Personality: (7.5/10) Gavin is talkative and friendly, but he is sometimes hesitant or awkward. He is afraid of spiders, and suffers from slight tendencies toward overindulgence. He always tries to do the right thing. He embraces racial tolerance, but he is less easy-going with regards to class and alignment. He will join any party, but his friendship and romance tracks are available to a more closely restricted pool of Bhaalspawn.

Quest? (3.5/5) Participation in two existing quests and one original one. While the existing quests can still be completed without Gavin in the party, the PC must have him in the party to complete the third quest. Completion of any of the quests are optional, but if Gavin is summoned to the temple to hear them, he will insist on going to hear what his superior wants.

Friendship? (3/5) Gavin has a friendship track with a neutral or good PC who is not an assassin.

Romance? (3.5/5) Gavin will romance a good PC who is not an assassin, bounty hunter, or necromancer if her wisdom is 10 or better and her charisma is 12 or better. A reputation of 11 or better is required to begin the romance. This requirement increases to 15 after a certain point late in the romance. The romance contains an optional flirt pack. Both the romance and the flirt pack contain mature content.

*ToB (BG2 only)? not applicable

Compatibility Considerations: (2/5) Gavin is not compatible with BGT at this time. Gavin is not known to be incompatible with any mod that may be applied to a Tutu installation. Gavin should not be installed on a BG2 installation prior to the Tutu conversion.

Technical Execution: (4/10) While Gavin is compatible with BG1 NPC Project, there is a bug in the romance track that fails to disable Gavin's romance once the PC commits to one of the romanceable males introduced by the BG1 NPC Project. Version 1.2 beta contains infrequent spelling and punctuation errors. Gavin's friendship and romance may be managed using Player-Initiated Dialogue.

Ease of Recruitment: (4/5) Easily recruited outside the Song of the Morning Temple outside Beregost. He will join any party.

Interaction with PC: (4/5) Depends on the alignment, class, gender and attributes of the PC. Gavin has friendship path with approximately 15 talks and a romance with 30 talks. Gavin has no PC-specific content for an evil PC or an assassin.

Interaction with BioWare NPCs: (4/5) Gavin has 1 to 6 banters with every BioWare NPC.

Interaction with Game World: (3/5) Frequent casual interaction with non-joinable NPCs in Beregost, moderate interaction with regards to game quests and plotline. His interjections are generally consistent with his class and alignment.

Integration with Game: (2/5) As the unmodified Baldur's Gate has little NPC interaction, Gavin clearly stands out as a mod NPC.

Crossmod Content: (4/5) Gavin has cross-mod content with the BG1 NPC Project, including romance conflicts with the three romanceable males in that mod. He has no cross-mod content with Finch, Indira, Mur'Neth, Xavia, Mulgore, or Jonathan.

*Maturity Rating: The romance portion of the mod contains some adult content. Assessment is based on sexual situations and violence. The friendship portion of the mod is on par with the original game.

Gameplay: (7/10) As a good-aligned cleric, Gavin is moderately powerful. He has good combat bonuses overall, though he does suffer a penalty to hit spiders. His high wisdom score gives him a significant spell bonus. Besides supportive, healing spellcasting, he has access to Boon of Lathander and Draw Upon Holy Might, both of which can enhance his combat abilities.

Subjective Pros: (3.5/5) Talkative NPC with a well-defined personality, provides a good-aligned divine spell-caster, and offers friendship as an alternative to a full romance. Gavin behaves consistently with his alignment and religion. He is very demonstrative about his romantic interest.

Subjective Cons: (-2.5/-5) Interjection frequency is significant, and might be considered a bit too high by some, and his stat total is higher than Viconia or Branwen. The friendship path is not as developed as the romance. Gavin's romance can be graphic at times, and offers disturbing consequences to some PC role-playing decisions. Not all the consequences of Gavin's past are pleasant for the romanced PC.

Overall: 49.5/80

Reviewer: berelinde
Favorite mods of same genre: Amber, Kivan, Xan

*Provided for information only, not included in rating

Edited by berelinde, 18 June 2007 - 05:23 AM.

[grogerson]

Nice example, berelinde. Though no review can be totally objective, you appear to have done the one thing to reduce subjectivity - used an objective template. If such a template is available while playing a mod, it would help in clarity and reducing spoilers as well.

If anyone is familiar with Ted Baer and his Movie Guide magazine, you can see how such objective templates can be used. Of course, such a guide can also kill interest in something - many of those who review for him soon have to stop. They become too sensitized to the bad/gratuitous content of the films they see. Of course the template is geared to a specific media type and focus audience, so such a template for mods must also state clearly what the focus group is. And moral/ethical content should also be considered ("Both the romance and the flirt pack contain mature content.").

Just remember, the idea of a template is to limit, not remove, subjective content. And a good reviewer will clearly distinguish observation from opinion.

Once again, thanks for the example, berelinde.

[berelinde]

Thanks for the opinion! However, while you were typing it, I was obliviously including yet more subjective material: evaluation on a point scale. Naturally, I'd appreciate comments on whether or not that is useful. I've used it as a supplement to the material provided earlier, not as a replacement.

My intention by including the points ratings was to allow the reader to see the extent to which each of these factors influenced reviewer perception of the mod. For example, Gavin is incompatible with BGT, which means a huge chunk of the gaming population that uses BG1 conversions can't even use the mod. That is an automatic 50% penalty.

Edited by berelinde, 18 June 2007 - 05:29 AM.

[Kulyok]

*SPOILERS*

(if it was a Gavin review thread, I wouldn't bring a spoiler tag. But a spoiler tag in the review itself might not go amiss).









I admit I found it somewhat difficult to read. Perhaps (shorter) tags in bold might help. Also, I find it more useful when all romance info and all friendship info(length of the friendship track, etc) is grouped together.

What I found missing(I did play the mod more than once):

- Quests aren't outlined at all, it is not mentioned that the first and the second quest are BG1 original quests;
names and areas are not given. Which is important, since the player will expect three completely new quests, and only will receive one.

- One of the most significant aspects of Gavin's personality is not defined - that he has abandoned his long-time partner because of her cheating, that he has a (foster) child, and that he, for a long while, does not mention it to PC, whether in romance or during friendship.

(To me, it's like reviewing Kivan and not mentioning Deheriana.)

- No mention of extensive romance conflicts with Ajantis, Coran and Xan.

- Flirt pack is both PC-initiated and NPC-initiated; it is also not mentioned how often flirts change during the romance progression.

- Meetings with Gavin's family members are not mentioned.

- Most importantly, it is not mentioned in the very first line that it is a TUTU mod.


Again, I must say I found it difficult to get through the formatting - my eyes hurt as I do.

[berelinde]

*SPOILERS*

(if it was a Gavin review thread, I wouldn't bring a spoiler tag. But a spoiler tag in the review itself might not go amiss).









I admit I found it somewhat difficult to read. Perhaps (shorter) tags in bold might help. Also, I find it more useful when all romance info and all friendship info(length of the friendship track, etc) is grouped together.

Fair enough. Will consider the viewpoint.

- Quests aren't outlined at all, it is not mentioned that the first and the second quest are BG1 original quests;
names and areas are not given. Which is important, since the player will expect three completely new quests, and only will receive one.

"Participation in two existing quests and one original one." Does that mean something other than that he will participate in two existing quests and has one new one?

- One of the most significant aspects of Gavin's personality is not defined - that he has abandoned his long-time partner because of her cheating, that he has a (foster) child, and that he, for a long while, does not mention it to PC, whether in romance or during friendship.

Significant *to you*. While I realize that this one fact is more important to you than anything else the mod offers, it is not so important to everyone who plays the mod. I included that not all players will find his past pleasant. I would happily include a link to the discussion forum as part of the review form, but I would no sooner include this information in the review than I would utterly spoil the offputting conclusion of Xan's unbonded romance in SoA. If readers wish to be spoiled utterly, they should be free to do so, but I would rather they take the initiative than have the information thrust upon them, when they expected no more than a simple review.

Again, this could be included as part of the discussion that follows. BB Code tags help hide spoilers for those that have them enabled, although same-color text or spacing like you've done above is also OK, during the discussion.

- No mention of extensive romance conflicts with Ajantis, Coran and Xan.

Does anyone else think it would be helpful to mention the conflict content of NPC mods? Would you expect to see it as part of Kelsey's review, or Solaufein's?

- Flirt pack is both PC-initiated and NPC-initiated; it is also not mentioned how often flirts change during the romance progression.

Will consider including it.

- Meetings with Gavin's family members are not mentioned.

As the reviewer, I didn't really think it was terribly noteworthy. But that is part of the reason for commentary to follow the review.

More importantly, it is not mentioned in the very first line that it is a TUTU mod.

I suppose it might be helpful, although it does appear under compatibility considerations. If that were the case, I would expect to see "BG2," "BG2/BGT," "BG1/BGT/Tutu" for every review.

Will definitely improve the formatting, but in the interests of not having to change it too many times, I'll leave that until more folk weigh in on its usefulness. Then make all the changes at once and post the result.

Edited by berelinde, 18 June 2007 - 05:59 AM.

[grogerson]

Thanks for the opinion! However, while you were typing it, I was obliviously including yet more subjective material: evaluation on a point scale. Naturally, I'd appreciate comments on whether or not that is useful. I've used it as a supplement to the material provided earlier, not as a replacement.

My intention by including the points ratings was to allow the reader to see the extent to which each of these factors influenced reviewer perception of the mod. For example, Gavin is incompatible with BGT, which means a huge chunk of the gaming population that uses BG1 conversions can't even use the mod. That is an automatic 50% penalty.

So I see. At least you're clear that it's subjective. Anyone who reads it should understand this also. This is discussion only, picking brains for ideas. And, yes, compatability issues are a definite must, especially for BGT/Tutu installations. Known mod conflicts are also a good idea.

So far, you've done well in objectivity/subjectivity delineation, in just your update. Since spoilers can't completely be avoided, any such content should have its own section and be clearly stated, with perhaps a (subjective) low/medium/high rating for spoilerishness.

[Kulyok]

Significant *to you*. While I realize that this one fact is more important to you than anything else the mod offers, it is not so important to everyone who plays the mod.

No, it is not; do not put words in my mouth, please. I also wouldn't speak for the silent majority.

"Participation in two existing quests and one original one." Does that mean something other than that he will participate in two existing quests and has one new one?

It is difficult to understand what "existing" and "original" means in this context; besides, the information is lacking, as I said. I am struggling to find something in the review what I wouldn't find in the mod's readme, except that Gavin has much more equipment that a Bioware BG1 NPC normally does.

As the reviewer, I didn't really think it was terribly noteworthy.

Which isn't terribly objective.


In case these two links aren't known to everyone:
- Sorcerer's Place has a Mod Review forum:
http://www.sorcerers...?/forum/23.html
- Imrahil's reviews at Ironworks Forums:
http://www.ironworks...f=2;t=024694;p=

[berelinde]

Opinions noted. No promises.

Regarding starting equipment, it is listed in order to allow viewers to compare to BioWare NPCs and those modified by BG1 NPC Project. By comparison to Gavin's 3 healing potions and an antidote, if BG1 NPC Project is installed, Imoen comes equipped with a wand of missiles and potions of haste, Ajantis can have plate mail and a shield that grants him two stars in sword-and-shield fighting style, Xan has his unique enchanted moonblade that grants him an AC bonus (as per BioWare), and Kivan can obtain a unique +2 spear within moments of recruitment, as well as a unique magical bow later in the game. These statements are made only to offer a point of comparison, not to contradict your opinion. I will leave it up to the player to decide whether "Gavin has much more equipment that a Bioware BG1 NPC normally does." Especially if the player intends to install other mods. After all, Finch comes fully outfitted with weapons, armor, and a highly useful bookbag.

Ordinarily, I would not advise reviewing mods while they are still in beta, nor would I encourage any modder to attempt to review their own work. This was not intended as a serious review of the mod in question, only an example of the kinds of information I would find useful. Naturally, opinions will vary.

Please feel free to carry on discussion on the general format or the usefulness of the type of information provided, however discussion on the contents of the review itself should probably be conducted on his forum at the Gibberlings Three, as this is not an official mod review, just a template. The forum can be found through the link on the DL page.

Edited by berelinde, 18 June 2007 - 08:02 AM.

[Tassadar88]

Hi, seemed nice to me, I only wondered whether the "Ease of recruitment" is a good thing to include or to give points for- on the one hand, it is a bit spoilerish, but potentially interesting piece of information for a player, but on the other, I would not use it to rate the quality of a mod.
Apart from that, I also thought about giving extra points for the documentation included in the package - if it has a readme, its contents, whether either a walkthrough/faq/hints section is included, something like this. The idea of listing out individual compatible types of BG installation sounds very nice to me as well. And the last question - if giving points, would there be some sort of reviewer guide, so no matter how subjective, the reviewer could rely on some sort of universally accepted criteria, that would lead to an idea of how much to give/substract?

[berelinde]

Hey, good idea about the documentation! You are right: some mods have outstanding documentation, others not so much so.

I think that if I understand the last comment of your post correctly, you're saying that reviewers should be basing their subjective evaluations on the same scale, using the same evaluation criteria. That's not a bad idea but it is difficult to come up with a "standard" in such a diverse field. It would make little sense to evaluate a full content mod like Fade, for example, on the same scale as one-day Alassa.

But mulling over that has raised another valid point: Fade should be evaluated on the same scale as Kelsey, and Alassa should be evaluated on the same scale as Ghareth, but the scale used should be identified.

Add a line:

Mod type: NPC - full development
or
Mod type: NPC - one day

And ask reviewers to use a common scale.
For example, poor ease of recruitment/retention would include several of the following in combinaton:
recruitment location is inaccessible
window of opprotunity to recruit the NPC is small
NPC requires a substantial quest for recruitment
NPC requires that the party have a certain composition in terms of class, alignment
NPC requires the presence of a particular BioWare NPC
NPC is easily provoked to leave the party forever
Once dismissed, the NPC leaves forever
NPC may only be recruited as part of a pair or group, or requires that a party slot be made available later in the game

NPCs with higher "Ease of recruitment/retention" scores would have less of these elements.

It's also worthwhile to note that even if an NPC has a low score for ease of recruitment/retention, it's only going to really drag them down a few points, out of a possible 80.

And all that is for just one part of the review!

But it is worthwhile. Needs thought, and I think I have a "someone" or two in mind to help

Edited by berelinde, 18 June 2007 - 10:38 AM.

[Kulyok]

Why would someone want to bully everyone use the same scale and format? BG2 mods are based on BG2, more or less, but basing all reviews on one? To me it sounds not unlike "all mods should be based on TDD". Which, come to think of it, is not a bad idea, if you replace TDD with something on my choosing... see what I'm trying to say? No matter how good a template is, it is never a good idea to try and enforce it, or actively suggest it - in my eyes.

(Which is my source of discomfort towards, say, G3 packaging templates: I find some features, like the obligatory path finding/inconvenient readme formatting, irksome, but I'm afraid all future G3 mods will continue to do it, and not in all cases because of the author's preference, but because all G3 mods are done this way.)

[Tempest]

I think a standard template for npc mods is a good idea, myself. If you're just looking for one particular bit of info that concerns you (like checking to see if the mod has ToB content, or checking for compatability), it's always in the same place. Mod npc's in particular all tend to play by similar rules and concepts, with truly unique features being extremely unusual, such as Xan's ToB content for the bonded path, or Kivan's Deheriana (or whatever her name is... I can never spell it right) arc.

One suggestion, though: reviews should make a note if the character has a quest (or multiple quests) that the party is required to do once they join the party, either to progress the romance or just to keep them in the party, such as the kidnappings of Fade, Amber, and Tashia.