Script for combining multiple mods

[llamabeast]

I proudly present a perl script for combining multiple mods and eliminating any clashes. I expect it to be useful for games where you want to play with a number of mod nations which might otherwise clash. It's quite a big, lengthy script, and it has a lot of built-in knowledge of how mods work. So, barring any typos (always a possibility), I'm fairly sure it can handle anything that can be thrown at it without producing any bugs in the resulting megamod.

The usage is:

combineMods era mod1.dm mod2.dm ... modN.dm newModName

e.g. combineMods 2 TombKings.dm vaetti.dm MagicMod

This will produce a mod called MagicMod.dm which will include the Tomb Kings and the vaetti, both in the middle era.

The era has to be a number between 1 and 3 (1=early etc). All the mod nations included in the combined mod will be set to be in that era, so that they can play together.

You can combine any number of mods with this script, and it's also fine to include mods which modify vanilla nations, e.g. CBM.

For those who haven't used a perl script before: On Linux or, I think, Mac, you need to open a terminal and type "perl combineMods ..." where ... is all the parameters, as given above. On Windows I believe you need to download ActivePerl. I think it's free.

Here it is:

[Edratman]

WOW.

As someone who usually plays 5 to 10 mod nations at a time, this is fantastic. I've had to manually change numbers throughout and I lost the takeout menu on which I had scribbled the new numbers.

:up::up::up:

[Sombre]

Can it handle all the different shapes?

Firstshape, secondshape, tmpshape, forestshape, plainshape etc? Summon, call allies, domsummon?

What does it do with nametypes?

[llamabeast]

Yep yep, it can handle all of them. It can handle all commands in Edi's unofficial modding manual (which I am assuming is complete).

With nametypes, it handles them sensibly, i.e. it keeps track of nametypes from different mods, renumbers them and makes sure each unit is pointing to the correct nametype. However, it obviously needs to choose which vanilla nametypes to overwrite. Near the top of the mod is an ordered list of which nametypes it should overwrite. I chose something like 127, 128, 150, 149, 148, 147 etc.. Obviously it's not ideal to overwrite any apart from 127 and 128, but generally I think that it's unlikely to be noticeable.

[llamabeast]

Oh, hang on. firstshape etc all currently assume that you've already defined the monster you want to change to. But of course that won't always be true.

Hmm. I will have a think about that, and try to fix it this afternoon.

[Xietor]

Thanks Llamabeast!

[llamabeast]

Okay, I have it very nearly fixed, but have run out of time before some friends who are staying for the weekend turn up. I'll re-upload it on Sunday or Monday.

[Edratman]

I'm very interested in seeing how you handle firstshape. It seems to be quite a formidible task, at least to my limited knowledge about the subject.

[llamabeast]

Okay! Reuploaded at last! See the first post.

The new version is significantly more cunning, and once more I don't _know_ of anything it can't handle. Summons, sites, nametypes, it's all in. The script is now a beast at about 900 lines (many of which are repetitious to be fair).

Also, the resultant mod is somewhat self-documenting. If you right-click on it from the Dom3 "Mod preferences" screen it will list which mods it includes, along with the version number of each. If you look at the mod file you'll find it lists near the top the ID number ranges that it uses.

[Edratman]

Thank you very much. I plan on giving it a test spin tonight when I have more time.

Today is one of my "Give-in-to-the-old-ball-and-chain" days. All of us married chaps must concede a couple of them a year. It is a day dedicated to doing things where she offers up the idea (sic), and the only thing I have to do is add the time, labor and money. :re:

Not much different than all the threads where someone reads a book and then suggests that someone can feel free to make a mod nation based on his brillant thoughts. :down:

Well, actually there is a difference; I can ignore those threads forever without enduring relentless, painful nagging. :)

[Foodstamp]

This is brilliant and will go a long way to combining my MOM nations mods when I get to that point. Thank you very much!

[llamabeast]

Just updated the mod to make a small but important fix: Previously the script actually rather embarrassingly couldn't cope with more than one space between a command and a value (e.g. "#newmonster 2500" would have confused it). That's now fixed.

[DrPraetorious]

So I've been absent from the modding forum and am glad to see that the first thing I needed to do has been done already :).

Thanks, llamabeast. This a vast improvement over anything I ever put together, or was likely to put together.

[pyg]

I've been working on a tool (http://forum.shrapnelgames.com/showthread.php?t=42415) that includes functionality similar to your combineMods script. In fact I've been cribbing from your script in order to see what I was supposed to do. I missed all the makemonster and summon stuff the first time. DMG does an additional thing that combineMods does not, which is it also combines the images so you can easily package your combined mods when you are done. For demo/testing purposes I put together a mega mod (http://forum.shrapnelgames.com/showthread.php?p=678843) (~30k lines) that combines 20 of the mod nations and activates all the original nations. It modifies 623 monster ids, 20 nations, 48 magic sites, 159 weapons, 61 armors, and 995 tga files. The number of sprite files is actually the hard limit as more than 1000 crashes with *too many sprites*. I still haven't done anything with nametypes but I may follow your lead and overwrite some of the *unimportant* ones. Sorry if this sounds competitive as my intent is cooperative :)

[chrispedersen]

Wow! Awesome llama and pyg.

Llama, will this handle the question I asked in the ABM thread, ie.,

CBM.dm
CBMfixes.dm

So for example if the unit has the same unit number

cbm.dm
#selectmonster 1
...
#descr "foo"
#end

cbmfix.dm
#selectmonster 1
...
#descr "bah"
#end

So monster 1 will have description "bah"? If so this would/will be amazing and awesome!

[llamabeast]

Regrettably not chris. I don't know of any way to fix that problem unfortunately. I can see it's quite a bummer for you.

pyg - great stuff. I should think a mod like that might be really popular to let people see all the mod nations. I bet it was very pleasing to code too! Incidentally I would be inclined to set it in the middle era, where things are all averagy. You should post about it in the main forum, it might get some interest as a funky alternative to the other single age mods. It might be worth listing the mod nations and describing which ones are considered balanced though (basically Sombre's mods, Tomb Kings, Alugra and Haida Gwaii so far as I can remember).

[Gregstrom]

I've tried using this for the Modness game, but got a non-functional mod out of it. I was using ActivePerl, and got the following messages.

C:\Perl>perl -w combineMods 2 sitemod.dm antilarium4.dm CBcomplete_1.5.dm CPCS.dm Holy_War_Divine.dm Holy_War.dm mytheology4modness.dm modnessmod

Use of uninitialized value $totalModNations in subtraction (-) at combineMods line 120.
Name of new mod: modnessmod.dm
Use of uninitialized value $maxSpellCounter in numeric gt (>) at combineMods line 183, <MODFILE> line 5039.
Use of uninitialized value $replaceWith in substitution (s///) at combineMods line 816, <MODFILE> line 25557.
Use of uninitialized value $replaceWith in substitution (s///) at combineMods line 816, <MODFILE> line 25557.
All done!
Any ideas?

Error message was name2spell:no such spell!

[Zeldor]

I am getting that:

Use of uninitialized value $totalModNations in subtraction (-) at combineMods line 120.
Name of new mod: SuperMod.dm

Use of uninitialized value $maxSpellCounter in numeric gt (>) at combineMods line 183, <MODFILE> line 23.
Use of uninitialized value $replaceWith in substitution (s///) at combineMods line 816, <MODFILE> line 12748.

[Zeldor]

Hmm... in my log file big part of CBM is highlighted in red, not in grey, like other mods and other part of cbm, the one that is being harvested. Maybe that's the reason? Or just my display? It is triggered by ' character. And in openoffice reader it is shown as unrecognised character. But it may be just my properties of text editor. I remember that you had issues on llamaserver with ' nations :)

[Zeldor]

Huh, it works great for CBM, it makes all damage just fine. In CPCS.dm it changes #damage 2 to #damage llamaspell1 etc. And I don't see anything special about CPCS mod to explain it.

[llamabeast]

I will have a quick look when I get home. Maybe I somehow uploaded a broken version or something, or maybe CPCS has some features I didn't take account of.

[Zeldor]

It must be CPCS, but dunno what. EVen leaving 1 spell from CPCS makes it crash in that way. So you can look just there.

[llamabeast]

Okay, bug hunted!!

I will upload the fixed version in a minute. However, all that needs to change is line 852 (it may be +-1, if I have unknowingly added or deleted a line).

It did read:
for my $modCounter (keys %unknownNationNumberFillIns) {
It should read:
for my $modCounter (keys %spellFillIns) {

D'oh!

The error messages are in fact just warnings. I had warnings turned off (bad llama!) and so hadn't seen them. In fact they do not indicate any actual problems and are harmless.

By the way, I really appreciate that you made the effort to get ActivePerl, work out how to run the code, print the error messages and so forth. Sorry it didn't work right away. I hope it works for you now!

[Gregstrom]

It works!

A thousand thankyous, llamabeast.

[Gregstrom]

I've found another issue, I'm afraid.

When renumbering of units takes place, it appears that new sites that have those units as recruitables don't get their #com and #mon renumbered appropriately. At least, you don't if you're combining the Antilarium4 mod.

[llamabeast]

Poo. When I wrote this I tested it really carefully, but I think I tested it on rather different mods - just CBM and nation mods if I remember correctly.

Unfortunately I am out at a big posh dinner tonight (by some quirk I seem to be on the records as a "lecturer" just for having done a little teaching last year so I get to go to a black tie feast!). I'll fix it tomorrow night, but I'm sorry in the meantime for holding your game up. Which mods does it affect (I am in the library now so can't check things properly)? Possibly if it is just say the magic site mod and the holy mod, you could combine all of the others, and keep those ones separate. They are unlikely to clash with the combined ones because the combined ones are given very high numbers for things IIRC.

[Gregstrom]

It affected Antilarium4.dm, but I've renumbered the #coms and #mons manually - there weren't too many of them.

[Gregstrom]

...apparently the script doesn't always get new summon spells correct either. In the unsanity combi-mod it looked like it got the Mytheology summons correct (at least, to a casual inspection), but apparently it messed up Divine War.

[llamabeast]

Bah, humbug! Sorry Gregstrom! I will try to take a look tomorrow night. By the sound of it it is a bug rather than a missing feature, so hopefully it will be quickish to solve.

[Amonchakad]

I'd really like to use this script to merge some mods,but I haven't got the slightiest clue on how to use it: I've never used perl,and I've tried downloading that perl program for windows,but it's a huge mess, so I went on another pc who has Ubuntu, and I tried your instructions for Linux, but it just says something like "no file found".
I'm a total newbie of Linux, so...does the combineMods file need to be in a particular directory? Do the mods need to be in the same one? Anything else that you might explain for this program to work?:D

[Amonchakad]

After learning a little on Ubuntu and Perl I managed to make the program work...awesome job you did there,Llama!
Just one thing...about the need to set a single age,can it be manually modified in the final mod file to have,for example,some of the new nations in early,some in middle,and some in late?

[llamabeast]

Great, good job on working out how to use it! The ease of running things like perl scripts on linux is a big plus to my mind.

Yes, it's very easy to change what era nations appear in. The mod command is #era, 1 is early, 2 is middle and 3 is late. So just search for the #era lines in the finished mod and change them however you'd like.

[Amonchakad]

After using your script to merge 6 mods, I've noticed that, while it reorders the unit IDs, it doesn't change the ID of the units recruited from sites(they still use the ID from the original mod), so I had to manually change the IDs in the site description.
I suppose it's a bug?:confused:
If it is,is it fixable?

[llamabeast]

It is indeed a bug, or rather an omission. In writing the script I had to try to think of every possible thing that there might be in a mod, and write a bit of code for it. I thought I had got everything (in my first released version I missed *loads* of stuff), but it turns out that I missed this. The Unsanity game hit the same problem, and also fixed it by hand. I can fix it easily enough, it will just be a couple of weeks before I have time probably (I am in mega PhD crunch time at the moment, and clearly shouldn't be on the forums!).

[lch]

Amonchakad, there's another piece of software, this time coded in Python, that does similar things like this Perl script, concerning combining nation mods only(?) by pyg, in case you want to try it out:

http://forum.shrapnelgames.com/showthread.php?t=42530
http://forum.shrapnelgames.com/showthread.php?t=42415

It requires a bit of programming knowledge from the user to use it, but it's wonderful code and a very interesting concept to create multi-nation mods or games.

[Zeldor]

I am getting lots of that "#damage llamaspell15_" etc stuff, trying to join few mods, including Holy War, any ideas?

[chrispedersen]

hey llama - any chance of a windows executable....
I'd like to make a bat program that allows you to choose portions of .dm files.

to do this, I need to be able to selectively (and repetetively) renumber. I'd like to use your program to do this.

However, active perl is too big to distribute; and to difficult for the casual user..

[llamabeast]

No, I don't know a way to make a stand-alone executable.

[Stavis_L]

No, I don't know a way to make a stand-alone executable.

Disclaimer: I've never done anything with Perl.

However, this page http://perldoc.perl.org/perlcompile.html

...indicates that Perl comes with a module that can spit out C code, which can then be compiled into a standalone executable.

[Amonchakad]

Llama, did you fix the site recruitables bug? I'm once again trying to merge some mods, and I wanted to know if your script could be used:)

[llamabeast]

Alas, no, it's still not quite right. I would say it is *likely* that I will fix it this weekend, but then recently I have found myself to be very unreliable.

[Gregstrom]

I shall now wait with bated breath.

[Sombre]

I'm bumping this fine thread because I too am interested in using the script, but worry it might omit a few things still.

[pyg]

New, improved, and in python. Check out this thread (http://forum.shrapnelgames.com/showthread.php?t=44889) for details.