This explains the basics of the 'struct paradigm', an alternative way of editing IE files by loading them into a WEIDU data structure, editing that structure, and writing it back.
A quick reminder: 'struct' is my name for a collection of WEIDU integer and string variables with a common prefix, like 'm_whatever'. An array ranges over a struct but you can have struct elements that aren't ranged over by any array.
Suppose you want to design a new spell, Edwin's Overpowered Fireball. It's based on Fireball, but it's a 4th level spell that does 1d8 damage per level if you're 10th level or lower, and 1d10 damage per level if you're level 11 or higher (maximum level 20).
The basic way you'd do this in WEIDU is to COPY_EXISTING the original Fireball spell to a new spell, and make the edits you need to turn it into the new spell. Each of those edits is, at its core, a byte-level edit to the unstructured mass of data that - as far as WEIDU is concerned - comprises the Fireball spell file. For instance, to change the description you'd do SAY 0x8 ~Edwin's Overpowered Fireball~. WEIDU doesn't know anything about the fact that the four bytes at 0x8 are actually the strref for the spell - all it knows is that there's four bytes of data there that you want to overwrite.
The limitations of this become particularly clear when you start editing the headers and extended headers of a file. You have to look up the slots in the file which record where the headers are stored in the raw data, do some math to calculate where the particular header you want lives, and then do some more math to translate header-relative locations to absolute locations. If you're planning not just to edit a header but to add or delete headers, then you have to adjust the entire file so as to update the header-location data for every header. And each new edit requires a new adjustment.
Modern WEIDU hides a fair amount of this from the end user. Functions like CLONE_EFFECT do all the calculations for you and just present you with a user-friendly interface. But under the hood, WEIDU's basic editing paradigm is unchanged. And as soon as you step outside the specific functionality offered by WEIDU's function library, you end up back having to do those manual lookups and calculations.
Most modern programming languages do things differently. What happens is that the file is loaded into some kind of data structure (an array or similar), which stores not only the raw data but its logical structure. The structure is then edited, and finally written back out. (If you've read chapter 2.4, on lib_2da, you'll have seen the same distinction made there.) The calculations are made once and for all when the file is read in (and again when it's written back to file), and all intermediate manipulations are of high-level, structured data. The 'struct paradigm' implements this in WEIDU, reading most common IE files into a WEIDU struct. This can make for drastically more readable code; it also makes it drastically easier to do tasks that don't happen to fall under the ambit of a predefined WEIDU function.
I need to flag one limitation of the struct paradigm. WEIDU is not natively designed to work this way, and that means there is a run-time overhead: struct-paradigm edits are several times slower than edits using WEIDU's native commands and functions. Often this doesn't matter (it's still a fraction of a second per file) but if you're making lots of small edits, it can add up. (In particular, it is almost never a good idea to combine struct-paradigm editing with COPY_EXISTING_REGEXP - though generally you should be cautious even using WEIDU functions with COPY_EXISTING_REGEXP on performance grounds). For what it's worth, my own approach is usually to use ADD_EFFECT and friends when something can be done straightforwardly with them, and the struct paradigm otherwise.
(For extremely complicated edits the struct paradigm can actually be quicker than the alternatives: its runtime consists almost entirely of the cost to read the file into a struct and write it again, whereas WEIDU's core paradigm takes a fixed time per task performed. I'll be honest, though: this isn't all that common.)
It's going to take a bit of work to systematically explain the struct paradigm, and I'm concerned it might be off-putting. So here's a sneak preview of how it works: the code to make Edwin's Overpowered Fireball. This code uses SFO syntactic sugar (see chapter 2.2) and will only work if included via lib_include.
spl.copy[%WIZARD_FIREBALL%=>WIZARD_EDWIN_FIREBALL]
[
// update header
m_name:=@1
m_description:=@2
m_level=4
// adjust casting time and power
m.ab.alter{s_casting_time=4}
m.ab_fx.alter{s_power=4}
// add new spell headers at levels 11-20
m.ab.clone{s_level=entry_index+11|number:i=10 match="s_level=10"}
// set damage
m.ab_fx.alter{s_dicesize=(p_level<11?8:10) s_dicenumber=(p_level=1?5:p_level)|match="s_opcode=12"}
]
(The automatic assignment of a SPWI code to the 'WIZARD_EDWIN_FIREBALL' spell is a separate function to the core struct paradigm, discussed in xx.)
LAF struct_read STR_VAR file="spwi304.spl" RET_ARRAY str=struct END
Key | Type |
---|---|
name | Strref |
name_unused | Strref |
completion_sound | String (8 bytes) |
break_sanctuary | Boolean |
This lists all the data stored for the (main header part of) the file. These are all stored as struct entries; for instance, when SPWI304.SPL is loaded, str_break_sanctuary=1.
Each stored item has a type, which determines how it is recorded. The (pretty much) full list of types is:
In each case, you can access the item as an integer or string entry in the struct, and can change it as usual via SPRINT or SET.
The full list of IE structure specifications known to SFO is available at %MOD_FOLDER%/sfo/doc/struct/struct_index.html. (It includes all the BG2/EE specifications for ARE, CRE, DLG, EFF, ITM, PRO, STO, SPL, and VVC files; it would be straightforward but tedious to add other specifications.) The documentation is generated automatically; if for some reason you need to regenerate it, just run the 'strdoc_document_all_strtypes' function.
SFO infers the struct type it's reading from the signature in the file; if for some reason you want to set it manually, you can do so via the 'strtype' STR_VAR. The syntax is (e.g.) strtype="are_v1" or strtype="cre_v1_1".
The inverse of struct_read is struct_write, which dumps the entire struct (not just any array tracking the struct) into the file. Again, normally the structure type is inferred, but you can override manually by setting strtype.
There are a few subtleties in how a struct is written:
With struct_read and struct_write in hand, we can start editing files. Here's code to copy the Fireball spell to a new innate spell, altering several features:
WITH_SCOPE BEGIN
COPY_EXISTING "%WIZARD_FIREBALL%.spl" "override/dw-infb.spl"
LPF struct_read RET_ARRAY m=struct END
SPRINT m_name @1
SPRINT m_description @2
SPRINT m_type Innate
SPRINT m_primary Necromancer
m_can_target_invisible=1
m_ignore_wild_surge=1
m_ignore_dead_magic=1
m_level=1
LPF struct_write STR_VAR struct=m END
BUT_ONLY
END
(The 'WITH_SCOPE' is because struct_read dumps a lot of data into a large number of strings in the struct, and unless we explicitly want to edit them elsewhere it makes sense to keep them confined.
There are also two so-called 'virtual' data types that can be written to; both abbreviate multiple datatypes. A 'commalist' is a string of entries separated by commas; a 'multiple' sends the same string to multiple locations in file. Here's code to edit the name, abilities, and saving throws of a creature:
WITH_SCOPE BEGIN
COPY_EXISTING "firarc01.cre" "override.cre"
LPF struct_read RET_ARRAY m=struct END
SPRINT m_both_names @1
SPRINT m_abils "18/76,6,10,14,16,8"
SPRINT m_saves "16,15,10,12,14"
LPF struct_write STR_VAR struct=m END
BUT_ONLY
END
struct_write carries an optional INT_VAR (boolean) argument, edit_strrefs_in_place. If this is set to 1, any strref variable is not written to a new strref; instead, the existing strref in the slot being edited is altered to match the strref variable. (For instance, if editing a spell description, edit_strrefs_in_place=1 will normally ensure that the scroll description is automatically changed to match.)
In most cases, it is easier to edit files using the struct_edit function (and its friends, struct_copy and struct_make, which are discussed below). Here's a simple example (we'll soon see how to do this more briefly).
DEFINE_PATCH_FUNCTION patch_firarc RET_ARRAY m BEGIN
SPRINT m_both_names @1
SPRINT m_abils "18/76,6,10,14,16,8"
SPRINT m_saves "16,15,10,12,14"
END
LAF struct_edit INT_VAR edit_strrefs_in_place=1 STR_VAR file=firarc01.cre edits=patch_firarc END
Here, the file 'firarc01.cre' is read into struct m using struct_read. The function 'patch_firarc' is then run, and returns m as an array. Then m is written back into the struct. (The choice of 'm' is hardcoded.)
SFO comes with various file-type-specific wraps for lib_struct. For instance, cre_edit basically just calls struct_edit with the extension fixed to be 'cre'. Similarly, we have are_edit, sto_edit, and so forth. (Some versions of these edit functions have extra functionality, as discussed below.) So we can do the same edit like this (taking patch_firarc as already defined):
LAF cre_edit INT_VAR edit_strrefs_in_place=1 STR_VAR file=firarc01 edits=patch_firarc END
This probably doesn't look like much of an improvement on just using struct_read and struct_write (or just the normal WEIDU paradigm!) There are two further steps that greatly simplify it. The first is that we can use the anonymous function construct (which in this context automatically includes a RET_ARRAY m), like so:
LAF struct_edit
INT_VAR edit_strrefs_in_place=1
STR_VAR file=firarc01.cre
edits=~
m_both_names:=@1
m_abils:="18/76,6,10,14,16,8"
m_saves:="16,15,10,12,14"
~
END
(Here we're also using a bit of SFO syntactic sugar to abbreviate the SPRINT commands.)
The second step is to use SFO's syntactic sugar for struct_edit itself (or, rather, for the wrap functions like cre_edit), like this:
cre.edit[firarc01|edit_strrefs_in_place:i=1]
[
m_both_names:=@1
m_abils:="18/76,6,10,14,16,8"
m_saves:="16,15,10,12,14"
]
This is getting a bit more manageable. Here's the syntax: 'ext.edit[myfile][myedits]' abbreviates 'LAF ext_edit STR_VAR file=myfile edits=myedits END'. Any additional arguments to the function 'ext_edit' are placed after the file name, separated by '|', as in the edit_strrefs_in_place argument above. (Note the ':i' to mark edit_strrefs_in_place as an INT_VAR; STR_VARs don't require any such marker.)
Three other useful features of struct_edit (and its wraps): firstly, you can use the usual SFO 'location', 'locbase', 'path' way to tell SFO where the file you want to edit lives. (By default SFO assumes it's an in-game file, as with all the examples above). This code would edit a file in the workspace:
spl.edit[SPWI304|path="%workspace%"]
[
m_type:=Innate
m_primary:=Necromancer
m_can_target_invisible=1
m_ignore_wild_surge=1
]
(Assuming your mod is immutable you'll never want to use 'location' or 'locbase', but they do work.)
Secondly, the file being edited can be a space-separated list of files, as in:
cre.edit[minsc minsc2 minsc4 minsc6 minsc7]
[
m_class:=FIGHTER
m_kit:=BERSERKER
]
Finally, SFO tries to do a bit of basic error-checking. If your anonymous function refers to 'm_something' and 'm_something' isn't in fact a valid field of this struct, it will throw a warning.
One quick bit of debugging advice: it is extremely easy to forget the 'm_' prefix, as in:
cre.edit[minsc minsc2 minsc4 minsc6 minsc7]
[
class:=FIGHTER
kit:=BERSERKER
]
This does nothing (the anonymous function sets the variables 'class' and 'kit' to those stated values, but they don't exit the function) and SFO's error-correction code can't catch it.
struct_make is a variant of struct_edit: instead of editing an existing file, it creates a new one. It has wrap functions just like struct_edit (cre_edit, spl_edit and the like) and uses the same syntactic-sugar syntax as struct_edit (you will normally want to access it this way). Normally SFO assigns default values of 0 to integer-valued datatypes and "" to string-valued datatypes, but in many cases there are other defaults. These are indicated in the documentation for each IE structure in [square brackets]: for instance, here is part of the documentation for the EFF structure.
Key | Type |
---|---|
opcode | Integer |
target[2] | Lookup (0=None, 1=Self, 2=PresetTarget, 3=Party, 4=Everyone, 5=EveryoneExceptParty, 6=CasterGroup, 7=TargetGroup, 8=EveryoneExceptSelf, 9=OriginalCaster) |
power | Integer |
parameter1 | Integer |
parameter2 | Integer |
parameter2a | Integer |
parameter2b | Integer |
timing | Lookup (0=InstantLimited, 1=InstantPermanent, 2=Equipped, 3=DelayLimited, 4=DelayPermanent, 5=DelayEquipped, 6=LimitedAfterDuration, 7=PermanentAfterDuration, 9=InstantPermanentAfterDeath) |
duration | Integer |
probability1[100] | Integer |
Most of these datatypes are initialized set to 0, but 'target' is initially set to 2 and 'probability1' is initially set to 100.
Here is some code to build an effect to do 3d6 fire damage with a save vs. spells for half at a -2 penalty:
eff.make[dw-fire]
[
m_opcode=12
m_parameter2b=8
m_timing=1
m_save_vs_spells=1
m_save_bonus="-4"
]
By default, files are created in the override directory; if you want them somewhere else, use location/locbase/path.
itm.copy[sw1h73=>dw-sword]
[
m_identified_name:=@1
m_identified_description:=@2
]
A few details of the struct_copy function:
Here is code to copy an item file into override from your mod, filling in its name and description:
itm.copy[dw-item1|source_location=item]
[
m_identified_name:=@1
m_identified_description:=@2
]
SFO stores extended headers as raw data: if you read ppinn01.sto into a struct m using struct_read, for instance, m_drink_0 is an 0x14-byte string containing the unprocessed data in the first 'drink' header in ppinn.sto. And m_drink_blockcount stores how many drink blocks there are in total.
(There is a further subtlety about how SFO stores extended headers when they are added or deleted; see the documentation if you're interested.)
In turn, SFO treats each extended header as an IE structure of its own: the 'drink' header in sto_v1, for instance, is IE structure 'sto_v1_drink'. In principle these can be edited manually with struct_read and struct_write: here is code to triple the prices of all the drinks in ppinn01.sto:
COPY_EXISTING "ppinn01.sto" override
LPF struct_read RET_ARRAY m=struct END
FOR (index=0;index<m_drink_blockcount;++index) BEGIN
SPRINT drink_struct EVAL "%m_drink_%index%%"
INNER_PATCH_SAVE "m_drink_%index%" "%drink_struct%" BEGIN
LPF struct_read STR_VAR strtype=sto_v1_drink RET_ARRAY s=struct END
s_price=s_price*3
LPF struct_write STR_VAR struct=s END
END
END
LPF struct_write STR_VAR struct=m END
BUT_ONLY
In practice you'll almost never want to do it this way.
The simple way to edit extended headers is with the struct_alter function (which you can think of as SFO's answer to ALTER_EFFECT). struct_alter will cycle through allextended headers of a given type, read it into a struct (hardcoded to be 's'), execute a (usually anonymous) function on it, and write it back again. Here's the same patch using struct_alter:
COPY_EXISTING "ppinn01.sto" override
LPF struct_read RET_ARRAY m=struct END
LPF struct_alter STR_VAR struct=m type=drink patch="s_price *=3" RET_ARRAY m=struct END
LPF struct_write STR_VAR struct=m END
BUT_ONLY
Here 's_price *=3' is actually an anonymous function specification and could, for instance, contain multiple commands.
struct_alter takes an optional argument, 'patch', which is again an anonymous function; it is first run on each struct and the patch is applied only if the match function returns value 1. (Normally the patch will just be a WEIDU logical expression). For instance, here's code that patches only drinks that cost >1 gold piece:
COPY_EXISTING "ppinn01.sto" override
LPF struct_read RET_ARRAY m=struct END
LPF struct_alter STR_VAR struct=m type=drink patch="s_price *=3" match="s_price>1" RET_ARRAY m=struct END
LPF struct_write STR_VAR struct=m END
BUT_ONLY
Note that 'match' does not make any permanent changes to the header it acts on: any temporary changes it makes are discarded.
Normally struct_alter itself will be used via SFO syntactic sugar. 'm.drink.alter{args}' abbreviates
LPF struct_alter STR_VAR struct=m type=drink patch=args END
so that, using also syntactic sugar for struct_edit, we can write our unconditional more-expensive-drinks patch as
sto.edit[ppinn01][m.drink.alter{s_price *=3}]
As usual with SFO's function-abbreviation sugar, other arguments (here, 'match') can be added, after a '|', in the argument of struct_alter: our conditional patch looks like
sto.edit[ppinn01]
[
m.drink.alter{s_price *=3|match="s_price>1"}
]
Again, SFO attempts some basic error catching: if you have a string 's_something' and s_something is not a data type in the struct, SFO will complain.
SFO provides struct functions to do each of these tasks - the SFO versions of DELETE_EFFECT, CLONE_EFFECT, ADD_ITEM_EFFECT and the like. At this point I'll just give their abbreviated syntax directly.
struct_delete takes a single argument, 'match', which defines an anonymous function that is run on each headers (reading each header into struct 's', as usual). Any headers for which the function returns '1' is deleted. If it has child headers (like the effect sub-headers of a spell or item ability header) these are deleted too. As with struct_edit, 'match' has no permanent effects on the header (except perhaps to mark it for deletion).
For instance, this code removes all but the first ability block on Fireball:
spl.edit[%WIZARD_FIREBALL%][m.ab.delete{s_level>1}]
struct_add adds a header (using the default values to fill its data types), and then runs the function 'patch' on it (reading the header into struct 's', as usual). Here's code to add a new drink to a store:
sto.edit[ppinn01]
[
m.drink.add{s_name:=@1 s_price=15 s_rumor_rate=6}
]
struct_add takes two optional INT_VAR arguments, 'insert_point' and 'number'. 'insert_point' specifies where in the existing headers the new header should be added: 0, for instance, puts it before heading 0, i.e. first. (By default, insert_point=-1, which means that the new header is added last). Here's code that puts the new drink second:
sto.edit[ppinn01]
[
m.drink.add{s_name:=@1 s_price=15 s_rumor_rate=6|insert_point:i=1}
]
'number' specifies how many entries should be added (the default is 1). struct_add also defines an internal variable, 'entry_index', which counts up from 0 and tracks which entry is being added; this allows you to make different patches to different added structures. For instance, here's code to add multiple items to a store:
sto.edit[ppinn01]
[
DEFINE_ARRAY item_list BEGIN blun01 blun02 potn14 potn15 sw1h05 END
m.item.add{s_number_in_stock=5 s_resref:=$item_list("%entry_index%")|number:i=5}
]
struct_clone copies an extended header. The syntax is very similar to struct_edit: On each header, an (optional) function 'match' is implemented and if its return value is 1 (or if no match function is given) then the header is copied (along with any child headers it has, e.g. the associated effects of a spell or item header) and the function 'patch' is applied to the copy. In each case the header is unpacked into the struct 's'.
For instance, here's code that takes the Ring of Fire Control (which grants +50% resistance to fire) so that it also grants 50% resistance to cold.
itm.edit[ring27]
[
m.fx.clone{s_opcode=28|match="s_opcode=30"}
]
By default, new headers are added immediately after the original header; if you set the INT_VAR clone_above to 1, the new header is instead added immediately before the original header.
You can use the same format as for struct_add to make multiple copies. Here's code that clones the level 1 block from fireball to levels 11-20:
spl.edit[%WIZARD_FIREBALL%]
[
m.ab.clone{s_level=entry_index+11|match="s_level=1" number:i=10}
]
SFO treats child headers as a sort of 'virtual header type'. For instance, the effect child headers of spell or item abilities have type 'ab_fx', whereas the casting-effect or on-equip effects are just type fx. You can then use struct_alter, struct_clone, struct_add and struct_delete to manipulate the child headers. For instance, here's code to make the saving throw penalty for Symbol: Stun -3:
spl.edit[%WIZARD_SYMBOL_STUN%]
[
m.ab_fx.alter{s_savebonus="-3"|match="s_save_vs_spell=1"}
]
The struct_ functions have two extra features when used to edit child headers. Firstly, the parent header is unpacked into the struct 'p' and can then be accessed. Here's code to extend Fireball to level 20:
spl.edit[%WIZARD_FIREBALL%]
[
m.ab.clone{s_level=entry_index+11|match="s_level=1" number:i=10}
m.ab_fx.alter{s_dicenumber = (p_level=1?5:p_level)|match="s_opcode=12"}
]
Secondly, each function has a new STR_VAR argument, match_parent. You can use it to specify a (normally anonymous) function that acts on the parent header (unpacked into struct 'p'). The action will be carried out only if the parent is matched.
The spl_make and spl_copy functions have some extra functionality.
SFO determines whether an argument is supposed to be an IDS entry heuristically (basically, it assumes an IDS entry if it has more than 8 characters), and assigns SPPR, SPWI or SPCL based on the spell type (in the file). Setting the INT_VAR is_ids to 1 or 0 overrides this. You can also set the STR_VAR 'type' to 'wizard', 'priest', 'innate' or 'class' to force the spell to resolve in (respectively) SPWI, SPPR, SPIN or SPCL (one useful reason to do this is to force a spell into the SPCL namespace rather than the SPIN namespace.
The resref of the new spell is returned by the function as the variable 'spell_resref' (also as 'value'). SFO's abbreviated syntax offers this way to collect that value:
new_resref=spl.copy[%WIZARD_FIREBALL%=>WIZARD_EDWIN_FIREBALL]
[
...
]
The new spell will be assigned a resref in the SPWI... namespace, and new_resref will be set to that resref. ('new_resref' can be anything you like; setting WIZARD_EDWIN_FIREBALL itself to the resref is often a good plan.)
>If spl_copy or spl_make is used to place a spell in the SPWI or SPPR namespace, SFO will by default set the BAM entries for the spell to the standard naming conventions: if the spell is SPPR345, say, then the icons will be set to SPPR345B and SPPR345C, as appropriate. (You can disable this by setting the INT_VAR standard_icons to zero).
You can also tell SFO to look for the BAM files themselves, and copy them to the appropriate new names. The variable 'icon_base_name' says what filename we should look for: if icon_base_name is set to myicon, SFO will look for MYICONA, MYICONB, and MYICONC. It will look in the location specified by the variables icon_location, icon_locbase and icon_path, using normal SFO conventions.
For instance, in the following code
spl.copy[%WIZARD_FIREBALL%=>WIZARD_EDWIN_FIREBALL|icon_base_name=edwin_fireball icon_location=resource icon_locbase=shared]
[
...
]
then SFO will look for the files edwin_fireballa.bam, edwin_fireballb.bam, and edwin_fireballc.bam in %MOD_FOLDER%/shared/resource.
If spl_copy or spl_make is used to place a spell in the SPWI or SPPR namespace, by default SFO will create a scroll for the spell. (You can override this default by setting the INT_VAR create_scroll to 0.) The scroll will match the spell's usability and targeting; its price is set by the table 'scroll_prices.2da' in %MOD_FOLDER%/sfo/data, and its name will follow CamDawg's naming conventions: the spell SPWIabc will have scroll CDIAabc and the spell SPPRabc will have scroll CDIDabc. The BAM file for the scroll is assumed to have the format '[spell_resref]A', as per the game's normal conventions, e.g. SPWI304A. (Note the interplay with the previous section on icons.)
The resref of the scroll is returned as the 'scroll_resref' argument (also 'scroll'); SFO's abbreviated syntax lets you return it like this:
new_resref,new_scroll_resref=spl.copy[%WIZARD_FIREBALL%=>WIZARD_EDWIN_FIREBALL]
[
...
]
You can also make spells manually, using the spl_make_scroll function, and doing so gives you some more options, e.g. to set the name and price directly. See the main documentation for details.