Exploring Monster Taming Mechanics in Final Fantasy XIII-2: The Remaining Tables

Continuing this miniseries of exploring the monster taming mechanics of Final Fantasy XIII-2, we'll finish off the remaining database tables that we want for the data relevant to monster taming. In the last article, we filled in a second table of passive abilities and connected those abilities to the monsters that had them through references in the monster table that used foreign keys to the ability table. In the first article, we had identified four tables besides the monster table that we would need as well, these being abilities, game areas, monster materials, and monster characteristics. We did the passive abilities table, but we still need a table for role abilities. In addition to this role ability table, we'll finish off the game areas, monster materials, and monster characteristics tables. These tables are all small, so we should be able to get through them without much effort.

The Monster Role Abilities

Unlike the passive monster abilities, of which there were 218, we only have 57 role abilities to worry about, and they can be found in this FAQ from Krystal109 on GameFAQs.com. Since these abilities are already in two HTML tables, it's easy to copy-and-paste them directly into a spreadsheet and tweak it the way we want it. I used Google sheets, and removed the "Best Infusion Source," "Learned by," and "Comments" columns because we don't need them. Then I split the merged cells in the "Role" column and copied the role names as necessary. Finally, I added a column for whether the ability is infusable or not, depending on which of the two tables it came from, before exporting the sheet to a .csv file.

But wait! There are actually 70 infusable role abilities in the Monster Infusion FAQ that we've been using, plus the non-infusable role abilities from the second HTML table, so we're missing a few. We are dealing with the common occurrence of incomplete data here. I had to cross-check the lists and add in the missing role abilities from the Monster Infusion FAQ. I ended up with 87 abilities in the end. This manual process is still probably faster than writing a script, but we're not sure, yet, if we have all of the non-infusable role abilities. We'll find out in a minute when we try to link up this data with the monster table.

Before we can add these associations, we have to generate a model of the role abilities table in Rails, like so:

$ rails g model RoleAbility name:string role:string infusable:boolean

This creates a migration that's all ready to run:

class CreateRoleAbilities < ActiveRecord::Migration[6.0]
  def change
    create_table :role_abilities do |t|
      t.string :name
      t.string :role
      t.boolean :infusable

      t.timestamps
    end
  end
end

Now we can add the import of the role abilities to the seeds.rb script right after the passive abilities import:

# passive abilities import from monster_abilities.csv
# ...

csv_file_path = 'db/monster_role_abilities.csv'

CSV.foreach(csv_file_path, {headers: true}) do |row|
  role_ability = row.to_hash
  role_ability['infusable'] = (role_ability['infusable'] == 'true')
  RoleAbility.create!(role_ability)
  puts "#{row['name']} added!"
end

# monster data import from monsters.csv
# ...

This import looks almost exactly like the passive abilities import, except that we need to convert the 'true' and 'false' strings for the infusable attribute into boolean values to match that attribute's data type. Once we have the role abilities imported and available, we need to add the associations to the monster table. We do this in a very similar way to how we did the passive ability associations, but this time it's simpler because we don't have to worry about red-locked abilities with the same base name as other abilities or ranks that complicated the passive abilities. We just have to find each role ability in each monster's set of skills and associate them by assigning the role ability to the right skill, like so:

CSV.foreach(csv_file_path, {headers: true}) do |row|
  monster = row.to_hash

  # ... associate passive abilities ...

  monster.keys.select { |key| key.include? '_skill'  }.each do |key|
    if monster[key]
      monster[key] = RoleAbility.find_by(name: monster[key])
      if monster[key].nil?
        puts "ERROR: monster #{monster['name']} #{key} not found!"
        return
      end
      puts "Found #{key} #{monster[key].name}"
    end
  end

  Monster.create!(monster)
  puts "#{row['name']} added!"
end

Now, we can't run this seed script quite yet. We still don't have a database migration or the monster and role ability models set up to handle these associations, so let's do that tedious work. First, the monster table migration needs all of the skill attributes changed from t.string to t.references and add_foreign_key statements need to be added for each skill. There are about a hundred of these, so I'll just show an example of what they look like:

class CreateMonsters < ActiveRecord::Migration[6.0]
  def change
    create_table :monsters do |t|
      # ... all of the other monster attributes ...
      t.references :default_skill1
      t.references :default_skill2
      t.references :default_skill3
      t.references :default_skill4
      t.references :default_skill5
      t.references :default_skill6
      t.references :default_skill7
      t.references :default_skill8
      t.references :lv_02_skill
      t.references :lv_03_skill
      # ... etc ...
    end

    # ... passive ability foreign keys ...
    
    add_foreign_key :monsters, :role_abilities, column: :default_skill1_id, primary_key: :id
    add_foreign_key :monsters, :role_abilities, column: :default_skill2_id, primary_key: :id
    add_foreign_key :monsters, :role_abilities, column: :default_skill3_id, primary_key: :id
    add_foreign_key :monsters, :role_abilities, column: :default_skill4_id, primary_key: :id
    add_foreign_key :monsters, :role_abilities, column: :default_skill5_id, primary_key: :id
    add_foreign_key :monsters, :role_abilities, column: :default_skill6_id, primary_key: :id
    add_foreign_key :monsters, :role_abilities, column: :default_skill7_id, primary_key: :id
    add_foreign_key :monsters, :role_abilities, column: :default_skill8_id, primary_key: :id
    add_foreign_key :monsters, :role_abilities, column: :lv_02_skill_id, primary_key: :id
    add_foreign_key :monsters, :role_abilities, column: :lv_03_skill_id, primary_key: :id
    # ... etc ...
  end
end

With that done, we still need to add the role abilities to the monster model in app/models/monster.rb with belongs_to. Remember how it felt weird that we were saying a monster belongs to its abilities in the last article? Well, that's still the direction we want, so we'll do it again with role abilities, even though it sounds weird:

class Monster < ApplicationRecord
  # ... passive ability belongs_to declarations

  belongs_to :default_skill1, class_name: 'RoleAbility', optional: true
  belongs_to :default_skill2, class_name: 'RoleAbility', optional: true
  belongs_to :default_skill3, class_name: 'RoleAbility', optional: true
  belongs_to :default_skill4, class_name: 'RoleAbility', optional: true
  belongs_to :default_skill5, class_name: 'RoleAbility', optional: true
  belongs_to :default_skill6, class_name: 'RoleAbility', optional: true
  belongs_to :default_skill7, class_name: 'RoleAbility', optional: true
  belongs_to :default_skill8, class_name: 'RoleAbility', optional: true
  belongs_to :lv_02_skill, class_name: 'RoleAbility', optional: true
  belongs_to :lv_03_skill, class_name: 'RoleAbility', optional: true
  # ... etc ...
end

Finally, we need to add the has_many declarations to the role ability model (because each ability has many monsters, right?):

class RoleAbility < ApplicationRecord
  has_many :default_skill1_monsters, :class_name => 'Monster', :foreign_key => 'default_skill1'
  has_many :default_skill2_monsters, :class_name => 'Monster', :foreign_key => 'default_skill2'
  has_many :default_skill3_monsters, :class_name => 'Monster', :foreign_key => 'default_skill3'
  has_many :default_skill4_monsters, :class_name => 'Monster', :foreign_key => 'default_skill4'
  has_many :default_skill5_monsters, :class_name => 'Monster', :foreign_key => 'default_skill5'
  has_many :default_skill6_monsters, :class_name => 'Monster', :foreign_key => 'default_skill6'
  has_many :default_skill7_monsters, :class_name => 'Monster', :foreign_key => 'default_skill7'
  has_many :default_skill8_monsters, :class_name => 'Monster', :foreign_key => 'default_skill8'
  has_many :lv_02_skill_monsters, :class_name => 'Monster', :foreign_key => 'lv_02_skill'
  has_many :lv_03_skill_monsters, :class_name => 'Monster', :foreign_key => 'lv_03_skill'
  # ... etc ...
end

Whew! We're finally ready to purge the database, run the migration again, and reseed the database, but remember to rename the monster table migration so that the timestamp in the filename is after the new role ability migration because the monster table migration needs to run last:

$ mv <old monster migration name> <new monster migration name>
$ rails db:purge
$ rails db:migrate
$ rails db:seed

After running these commands, we find that a bunch of non-infusable role abilities were missing from the list. Most of these were basic Commando commands like Attack, Ruin, Launch, and Blitz, or variations on Saboteur commands like Dispel II or Heavy Dispelga. When all missing abilities are found and fixed, we should have 127 role abilities. Then there are two typos in the Monster Infusion FAQ monster list that need to be fixed as well: an instance of "Deprotectga" instead of "Deprotega" and an instance of "Medigaurd" instead of "Mediguard." With that task done, we can call the role ability table complete. There are six hidden role abilities, one for each role type, but they are never referenced in the monster attributes because they are fixed by the role type of the monster. By infusing 99 levels worth of monsters of the opposite role type, (e.g. commando and ravager are opposites) the monster will acquire its hidden role ability. It's questionable whether adding them to the database is useful, so let's move on to game areas.

Making a Game Location Graph

You start the game in New Bodhum 003 AF. Actually, you start in Valhalla, but after getting through the intro segment with Lightning and a bunch of over-the-top cut scenes, the game really starts in New Bodhum with Serah. As you progress through the game, you unlock more and more new areas by jumping through time gates. Some locations have multiple time gates that lead to multiple new locations. Different monsters live in different locations, and which locations they live in is noted as one or more of the monsters' location attributes.

We want to capture the graph of these location dependencies in a database table and link the locations to the monster attributes so that we can figure out the earliest possible times in the game that we can acquire monsters with certain abilities. In order to do that, we're going to build a table to represent that graph. We could build this table in a couple of ways. One way is to make an adjacency matrix, but this representation requires a row and column for each node in the graph. In this case such a matrix would be sparsely populated, so we're going to use an adjacency list instead. Each row in the table will have one location's name, and which location leads to this location. Since each location has only one other location as a source, we can get away with this simple table structure. There are only a small number of locations, so the .csv file for this table can be created by hand. Plus, I couldn't find a good list on the internet, so here it is, created from scratch:

name,source
New Bodhum 003 AF,nil
Bresha Ruins 005 AF,New Bodhum 003 AF
Bresha Ruins 300 AF,Bresha Ruins 005 AF
Yaschas Massif 110 AF,Bresha Ruins 300 AF
Yaschas Massif 010 AF,Bresha Ruins 005 AF
Oerba 200 AF,Yaschas Massif 010 AF
Yaschas Massif 01X AF,Oerba 200 AF
Augusta Tower 300 AF,Yaschas Massif 01X AF
Serendipity Year Unknown,Yaschas Massif 01X AF
Sunleth Waterscape 300 AF,Bresha Ruins 005 AF
Coliseum ??? AF,Sunleth Waterscape 300 AF
Archylte Steppe ??? AF,Sunleth Waterscape 300 AF
Vile Peaks 200 AF,Archylte Steppe ??? AF
Academia 400 AF,Sunleth Waterscape 300 AF
Yaschas Massif 100 AF,Academia 400 AF
Sunleth Waterscape 400 AF,Yaschas Massif 100 AF
Augusta Tower 200 AF,Academia 400 AF
Oerba 300 AF,Augusta Tower 200 AF
Oerba 400 AF,Oerba 300 AF
Academia 4XX AF,Augusta Tower 200 AF
The Void Beyond ??? AF,Academia 4XX AF
Vile Peaks 010 AF,Academia 4XX AF
A Dying World 700 AF,Academia 4XX AF
Bresha Ruins 100 AF,A Dying World 700 AF
New Bodhum 700 AF,A Dying World 700 AF
Academia 500 AF,New Bodhum 700 AF
Valhalla ??? AF,Academia 500 AF
Coliseum ??? AF (DLC),New Bodhum 003 AF
Valhalla ??? AF (DLC),New Bodhum 003 AF
Serendipity ??? AF (DLC),New Bodhum 003 AF

As we've already done with the other tables, the first thing we need to do to import this .csv file is generate the model for this location table:

$ rails g model Location name:string source:references

Notice that we created the source attribute as a reference right away, and this reference that's being created is different than the others that we created for passive and role abilities. Whereas those references were associated with new tables, this source reference is associated with the location table itself. To construct this self-reference, we don't actually have to do anything to the migration. Rails does the right thing by default, so the migration is simple:

class CreateLocations < ActiveRecord::Migration[6.0]
  def change
    create_table :locations do |t|
      t.string :name
      t.references :source

      t.timestamps
    end
  end
end

We also want to add references to the location attributes in the monster table, so we need to change that migration as well:

class CreateMonsters < ActiveRecord::Migration[6.0]
  def change
    create_table :monsters do |t|
      t.string :name
      t.string :role
      t.references :location, foreign_key: true
      t.references :location2, foreign_key: {to_table: :locations}
      t.references :location3, foreign_key: {to_table: :locations}
      # ... the mess of other attribute declarations ...
    end

    # ... a mess of foreign key declarations ...
  end
end

Notice that for these location references, the foreign key is specified directly in the attribute declaration. I recently noticed that this was possible, and it's much cleaner than adding foreign keys at the end, so I'm switching to doing it this way.

The next step is to update the seeds.rb script for the location and monster models. Let's do the models first this time. The location model will need a self-reference for the source attribute and declare that it has many monsters for the location attributes in the monster model, like so:

class Location < ApplicationRecord
  belongs_to :source, class_name: 'Location', optional: true
  has_many :location_monsters, :class_name => 'Monster', :foreign_key => 'location'
  has_many :location2_monsters, :class_name => 'Monster', :foreign_key => 'location2'
  has_many :location3_monsters, :class_name => 'Monster', :foreign_key => 'location3'
end

The self-reference doesn't need to declare its foreign key because it's the same name as the name of the reference. Rails can infer the foreign key correctly in this case. Now for the monster model:

class Monster < ApplicationRecord
  belongs_to :location, class_name: 'Location', optional: true
  belongs_to :location2, class_name: 'Location', optional: true
  belongs_to :location3, class_name: 'Location', optional: true
  # ... a mess of other reference declarations ...
end

It makes a little more sense that a monster belongs to a location, so that's nice, but it's really just words. It's the direction of the associations that's important, and belongs_to means the reference is in the current model and points to the Location model, which is what we want. Finally, we can update the seeds.rb script to import locations. This update happens in two parts. First, we need to read from the .csv file we created:

# ... role abilities import from monster_role_abilities.csv ...

csv_file_path = 'db/monster_locations.csv'

CSV.foreach(csv_file_path, {headers: true}) do |row|
  location = row.to_hash
  location['source'] = Location.find_by(name: location['source'])
  if location['source'].nil? && location['name'] != 'New Bodhum 003 AF'
    puts "ERROR: location #{location['source']} not found!"
    return
  end
  Location.create!(location)
  puts "Location #{location['name']} added!"
end

# ... monster data import from monsters.csv ...

For this import, we're not only populating the table, but we have to correctly assign the self-references. To do that assignment, we had to make sure when writing the .csv file that every location that was referenced in a source attribute appeared before that reference as a named location. That way, when this script looks for a location in the source attribute, it will find it already exists as a location in the table. If there were loops, this import would require two passes to find all of the location names first before assigning references, but there are no loops so lucky us. I also check that each source location is actually found, in case I made any typos, but there is one intentional nil source for where you start the game in New Bodhum 003 AF.  That particular nil source needs to be handled specially. With the location table filled in, we can associate the location attributes in the monster table:

CSV.foreach(csv_file_path, {headers: true}) do |row|
  monster = row.to_hash

  # ... associate role abilities ...

  monster.keys.select { |key| key.starts_with? 'location'  }.each do |key|
    if monster[key]
      monster[key] = Location.find_by(name: monster[key])
      if monster[key].nil?
        puts "ERROR: monster #{monster['name']} #{key} not found!"
        return
      end
      puts "Found #{key} '#{monster[key].name}'"
    end
  end

  Monster.create!(monster)
  puts "#{row['name']} added!"
end

This part of the script is exactly the same as it was for role abilities, just doing it for the location attributes instead. Now we can run this script again, making sure that the monster migration is still the most recent migration since it needs to be performed last:

$ mv <old monster migration name> <new monster migration name>
$ rails db:purge
$ rails db:migrate
$ rails db:seed

After running this script, I found a number of additional typos in the FAQ, so that data validation code is really coming in handy. I also caught an unexpected shortcut that the FAQ author took with locations. If a monster appears in two similar locations, they combined the years with a slash, e.g. "Bresha Ruins 100/300 AF." I had to split these out into two separate lines to adhere to our data model. Once that shortcut and the typos were fixed, the script ran to completion, and that's it for locations.

Wrapping up Monster Materials and Characteristics

This article is getting pretty long, so I'm going to finish off the monster materials and characteristics tables quickly. The characteristics is a straightforward table that doesn't require figuring out anything new. The materials table can also be straightforward, at least for now. The monster materials themselves constitute a simple table with a name, a grade of 1-5, and a type of either biological or mechanical. The trick is that we'll eventually want to know when in the game we can get each grade of monster material because they're used to upgrade our tamed monsters. The way to acquire those materials is by defeating other monsters in battle. The different materials are some of the loot that the monsters drop. This information is all in the Monster Infusion FAQ, but it would require another parser to extract it and import it into the monster table. We can do that later in the series, but right now I want to complete the tables and get on with starting to view the data so we'll just go with making the simple material table.

A table of the materials can be found in the same HTML FAQ from Krystal109. Like the first set of role abilities, we can copy this table into a spreadsheet and then export it to a .csv file. Then we run through the same steps of importing this data into our Rails database, starting with generating a material model:

$ rails g model Material name:string grade:integer material_type:string

The migration is created without need for modification, and right now, this will be a stand-alone table so there's no need to change any model code. All that's left is to import the .csv file in the seeds.rb script:

# ... location import from monster_locations.csv ...

csv_file_path = 'db/monster_materials.csv'

CSV.foreach(csv_file_path, {headers: true}) do |row|
  Material.create!(row.to_hash)
  puts "Material #{row['name']} added!"
end

# ... monster data import from monsters.csv ...

Then we can do the same thing with the monster characteristics table also found in the HTML FAQ from Krystal109. Do the same drill of copying the table into a spreadsheet and then exporting it to a .csv file before generating another model for this table:

$ rails g model Characteristic name:string description:string

Finally, add this data import to the seeds.rb script:

# ... material import from monster_materials.csv ...

csv_file_path = 'db/monster_characteristics.csv'

CSV.foreach(csv_file_path, {headers: true}) do |row|
  Characteristic.create!(row.to_hash)
  puts "Characteristic #{row['name']} added!"
end

# ... monster data import from monsters.csv ...

And we're ready to run the migration and import:

$ rails db:purge
$ rails db:migrate
$ rails db:seed

The purge is done first to clean out everything that we already had in the database. Otherwise, when we run it again now, we'll be duplicating all of the other data that we already imported in the seeds.rb script on previous runs. Running this sequence of commands will build a fresh database from scratch, and it should finish cleanly in one shot.

That last step will wrap up the database design and building, for now. We'll have a bit of work to do later when we want to connect the monster materials to the monsters that drop them, but this is good enough to get started with creating views of this data. After spending multiple articles parsing, exporting, importing, and creating data for the database, I'm anxious to get moving on those views, so that's what we'll start looking at next time.

Exploring Monster Taming Mechanics Table of Contents:

Part 1: Introduction

Part 2: Data Collection

Part 3: Data Validation and Database Import

Part 4: Relational Data

Part 5: The Remaining Tables

Part 6: Viewing Data

Part 7: Viewing More Data

Part 8: Viewing the Monster Data

Part 9: Viewing More Monster Data and Abilities

Part 10: Filtering Monsters

Part 11: Asking Deeper Questions

Part 12: Finding Monster Materials

Part 13: Monster Characteristics

Part 14: Dogfooding