tag:blogger.com,1999:blog-62277912520398555872024-03-14T10:52:09.806-05:00Lucid MeshMusings on software development, technology, and their interconnections with a programmer's everyday lifeSam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.comBlogger251125tag:blogger.com,1999:blog-6227791252039855587.post-45636782956796777832021-03-08T19:11:00.002-06:002021-03-20T15:31:34.834-05:00Playing with Gravity to Learn CUDA: Optimization<p>The CUDA gravity simulator has reached the point where it has a simulation engine that can <a href="https://sam-koblenski.blogspot.com/2021/01/playing-with-gravity-to-learn-cuda-n-body-simulation.html">display a 1024-body simulation</a> in real-time. (<a href="https://sam-koblenski.blogspot.com/2021/02/playing-gravity-learn-cuda-optimization.html">Check it out from the beginning.</a>) In getting to that point, we hit the limit on the number of threads that can be started in a single thread block in CUDA, but of course, that is not the end of the story. We can still increase the number of bodies in the simulation further, and after we've explored how to do that, we'll experiment with the parameters of the simulation to see if we can get anything interesting going that looks like a star cluster. Spoiler alert: we can, and we will.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdhykullW_qDyu6F3rg6YsZ8ruTG7s9I6LPcyBPexCnUFk2hsPleTrt7dctoFABiDRGa3hrG1JI4PTaWgtRJtSS3BfY9CijOub2kGPITmOidl6HcM2GPtDlrZ_UJ_mI-SCXuOnLwV8gUs/" style="margin-left: 1em; margin-right: 1em;"><img alt="Model of the Solar System" border="0" data-original-height="1019" data-original-width="1500" height="271" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdhykullW_qDyu6F3rg6YsZ8ruTG7s9I6LPcyBPexCnUFk2hsPleTrt7dctoFABiDRGa3hrG1JI4PTaWgtRJtSS3BfY9CijOub2kGPITmOidl6HcM2GPtDlrZ_UJ_mI-SCXuOnLwV8gUs/" width="400" /></a></div><br /><a name='more'></a><h4 style="text-align: left;">Pushing Up the Thread Count</h4><script>hljs.initHighlightingOnLoad();</script><div>In the last post we stopped at 1024 threads not because that was all the GeForce GTX 1050 card I'm using can handle, but because that's a hard limit in CUDA on the number of threads that can be started within a single block, regardless of which GPU you're using. However, we can have more than one block, so we can increase the number of blocks to further increase the number of threads running simultaneously and thus increase the number of individual bodies in the simulation without needing to change the kernel. The number of blocks we want to use is specified as the first parameter within the special <span style="font-family: courier;"><<< blocks, threads >>></span> syntax when calling a kernel, and so far we've always set <span style="font-family: courier;">blocks</span> equal to 1.</div><div><br /></div><div>Instead of leaving the number of threads at 1024 and increasing the number of blocks, we can further optimize the efficiency of the simulation by knowing a bit more about the CUDA execution model. In the execution model, when threads are executing on the GPU cores, the threads are organized into groups referred to as "warps." Each warp is a fixed group of 32 threads that execute on hardware resources in lock-step. Since the number of threads in a warp is fixed at 32, the number of cores on a GPU will be a multiple of 32 for everything to work out. For example, my GTX 1050 has 640 cores, which is 32x20. If we execute a number of threads that isn't a multiple of 32, then the last warp will contain some number of empty threads to fill up the warp to 32 threads, so we might as well simulate a multiple of 32 bodies to use the hardware most efficiently.</div><div><br /></div><div>We can also take advantage of the fact that thread execution is managed at the block level, and organizing the blocks into a small number of warps will give the GPU the most flexibility in assigning resources and hiding memory latency. If the kernel only does a small amount of computation compared to memory accesses (where the kernel needs to access memory for variables passed into the kernel), then a slightly higher number of warps will help keep the cores busy executing more threads while other threads are waiting for memory accesses. Multiple warps can be assigned to the same set of 32 cores and one warp will execute while another warp is waiting for memory. This warp-switching is supported by hardware and is extremely efficient. In our case, there's enough computation done in the kernel that one warp per block is enough to optimize the execution and hide the memory latency. This was not determined solely by code inspection and reasoning about the execution model, but by taking time measurements while the simulator was running, as shown in this code:</div>
<pre><code class="cpp">cudaError_t gravityIter(vec3d* d, vec3d* dev_v, vec3d* dev_d, const float* dev_m, const vec3d* dev_v0, const vec3d* dev_d0, float dt, unsigned int size) {
clock_t t = clock();
// Launch a kernel on the GPU with one thread for each body.
unsigned int blocks = size / 32;
gravityKernel <<< blocks, 32 >>> (dev_v, dev_d, dev_m, dev_v0, dev_d0, dt);
// cudaDeviceSynchronize waits for the kernel to finish
cudaDeviceSynchronize();
printf("t_iter = %d\n", clock() - t);
return cudaStatus;
}</code></pre>
This is the same code that was shown in previous posts that calls the <span style="font-family: courier;">gravityKernel()</span>, but the error checking was removed for clarity. Here, the number of blocks are determined by dividing the simulation size by 32, and the number of blocks and threads are passed to the kernel execution as <span style="font-family: courier;"><<< blocks, 32 >>></span>. The performance measurement is simply measuring the clock before the kernel starts and after all of the threads have synchronized, and printing out the time change. Lower is better, and there was no benefit to increasing the block size above 32. At the original value of 1024, kernel execution was actually less efficient because the blocks couldn't be managed well across the 640 cores in the GPU, so there is a tradeoff going on between memory latency and thread management for any given kernel.<div><br /></div><div>In order to use this blocks/threads setup, we also needed to make one change to the kernel code because the index into the various data arrays is no longer solely determined by the thread index. Now the block index comes into play as well, so we need to take that into account by calculating the overall index using the always-available <span style="font-family: courier;">blockIdx</span> and <span style="font-family: courier;">blockDim</span> variables as follows:<br />
<pre><code class="cpp"> int i = blockIdx.x * blockDim.x + threadIdx.x;</code></pre>
With this optimal thread organization determined, we can start scaling up the number of bodies again, and we can use the time measurement to figure out when enough is enough. We want the simulation to still look fairly smooth on the display, so targeting about 30fps would be desirable. That would mean kernel execution should take about 33msec to be optimal, and kernel execution will scale super-linearly with the number of bodies because each body needs to calculate its force on every other body in the system and we'll be well past one body per core. I found that for my GPU, 8 bodies per core, or 5,120 bodies total, still resulted in a smooth simulation and was pretty respectable for simulating a star cluster. Any more than that, and things visibly slowed down. The number of bodies should always be a multiple of the number of cores because if we were left with a group of bodies simulating on only a fraction of the available cores, we'd effectively have wasted the fraction of cores that are left empty for the last part of that simulation cycle. Remember, all threads in a warp execute in lock-step, all threads are executing the exact same code so they should finish at nearly the same time, and all threads are synchronized at the end of the kernel execution. That means all cores should finish thread execution at nearly the same time, so choosing the number of bodies as a multiple of the number of cores makes perfect sense.</div><div><br /></div><div><h4 style="text-align: left;">Experimenting with a Star Cluster</h4><div>Now that we have a reasonable number of bodies for a star cluster simulation, let's see what that looks like. We had left the simulation configuration at the scale of the solar system, but now that we're trying to simulate a star cluster, we need to expand that out substantially. Let's try a space extending about 100 light-years across. We need that in meters, and a light-year is 9.46e15 meters, so we'll use 9.46e17 for our scale. We can't continue to simulate in half-hour increments at this scale, or we'll be waiting forever for anything to happen. Increments of 1,000 years seems more appropriate. That gives us the following constants for our simulation:</div>
<pre><code class="cpp">const float _scale = 9.46e17;
const float _point_size = _scale / 50.0;
const int _num_bodies = 8 * 640;
const float _dt = 3600*24*365.25*1000;</code></pre>
Then we need to set up the stars in the system. Realistically, the <a href="https://en.wikipedia.org/wiki/Stellar_classification#Harvard_spectral_classification">distribution of stars</a> in a cluster (or anywhere in the universe) is biased towards red dwarf or M-class stars, which make up 75% of all stars, but are only 0.08-0.45 solar masses. Blue giant or O-class stars make up only a tiny fraction of all stars at 0.00003%, but are greater than 16 solar masses. Instead of trying to reproduce this distribution, we'll take the easy way out and model a uniform distribution of 0.5-1.5 solar masses for our star cluster. It should be sufficient for this exploration, and it's simple to understand and code. We can set up this star configuration in <span style="font-family: courier;">initBodies()</span> as follows:<br />
<pre><code class="cpp">void initBodies(float* m, vec3d* v, vec3d* d) {
// set origin
m[0] = 0.0;
d[0] = { 0, 0, 0, 1 };
v[0] = { 0, 0, 0, 1 };
float nominal_mass = 2.0e30;
for (int i = 1; i < _num_bodies; i++) {
m[i] = (rand() / (float)RAND_MAX) * nominal_mass + nominal_mass / 2;
d[i].x = (rand() / (float)RAND_MAX - 0.5) * _scale * 2;
d[i].y = (rand() / (float)RAND_MAX - 0.5) * _scale * 2;
d[i].z = 0.0;
d[i].a = 1.0;
v[i].x = 0.0;
v[i].y = 0.0;
v[i].z = 0.0;
v[i].a = 1.0;
}
}</code></pre>
A solar mass is about 2.0e30 kg, and the mass calculation gives a uniform distribution of masses around 0.5-1.5 solar masses. The position calculation similarly gives a uniform distribution around <span style="font-family: courier;">+/-_scale</span> in both the x- and y-dimensions. Let's see what this simulation configuration looks like:<br /></div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="492" src="https://www.youtube.com/embed/pYxwS0HXzSk" width="592" youtube-src-id="pYxwS0HXzSk"></iframe></div><br /><div>It's not terribly interesting because nothing starts out in motion. It takes a very long time for gravity to have enough of an effect to really get things in motion, except for the few stars that were randomly placed very close together to begin with. Those stars are quickly attracted to each other, accelerating rapidly. Then, because there's no way to handle collisions, the stars will pass each other and go flying off in opposite directions. Over time more of the system will be set in motion, but it seems like it will mostly be ejections from the system and the result is not going to be realistic at all. Thinking about the initial conditions, at this point in star formation the system is definitely going to be in motion. We can start our refinements by trying to initialize each star with a random motion. Something like a maximum of 5 km/sec in either direction and dimension sounds reasonable:</div>
<pre><code class="cpp">void initBodies(float* m, vec3d* v, vec3d* d) {
// set origin
m[0] = 0.0;
d[0] = { 0, 0, 0, 1 };
v[0] = { 0, 0, 0, 1 };
float nominal_mass = 2.0e30;
for (int i = 1; i < _num_bodies; i++) {
m[i] = (rand() / (float)RAND_MAX) * nominal_mass + nominal_mass / 2;
d[i].x = (rand() / (float)RAND_MAX - 0.5) * _scale * 2;
d[i].y = (rand() / (float)RAND_MAX - 0.5) * _scale * 2;
d[i].z = 0.0;
d[i].a = 1.0;
v[i].x = (rand() / (float)RAND_MAX - 0.5) * 10000.0;
v[i].y = (rand() / (float)RAND_MAX - 0.5) * 10000.0;
v[i].z = 0.0;
v[i].a = 1.0;
}
}</code></pre>
A simulation of this configuration looks like this:<div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="492" src="https://www.youtube.com/embed/kRGHQrZ9Sv8" width="592" youtube-src-id="kRGHQrZ9Sv8"></iframe></div><div><br /></div><div>While this simulation looks more interesting—there's some motion to the system and ejections happen only very rarely—it's also not realistic. It looks more like a slow-motion particle simulation of a gas than a gravity simulation of a star cluster. It also doesn't have a discernable pattern to it or look like it will ever reach a state like the star cluster pictured at the top of this post. We need to force our simulation in the direction we want it to go with more intent.</div><div><br /></div><div><h4 style="text-align: left;">Setting up a (More) Realistic Star Cluster</h4><div>If we stop to think about what a real star cluster looks like, it does not have a uniform distribution of stars. The stars are concentrated more in the center of the cluster and the number of stars falls off the further from the center of the cluster they are. We can very easily model this kind of distribution by changing our coordinate system from the standard rectangular coordinates to polar coordinates. In polar coordinates, instead of having an <i>x</i> and <i>y</i> value, we have a radius <i>r</i> that represents the distance from the origin, and an angle <i>θ</i> that represents the angle from the x-axis. If we choose a random <i>r</i> and <i>θ</i> for each body with a uniform distribution, we will naturally end up with a distribution of points that's more concentrated in the center because about the same number of points should exist on concentric circles around the center. Then, we simply convert back to rectangular coordinates using the following equations:</div><div><br /></div><div style="text-align: center;"><i>x = r * cos(θ)</i></div><div style="text-align: center;"><i>y = r * sin(θ)</i></div><div><i><br /></i></div><div>We can implement the polar coordinate generation of stars in <span style="font-family: courier;">initBodies()</span> like so:</div>
<pre><code class="cpp">#define M_PI 3.14159265358979323846
void initBodies(float* m, vec3d* v, vec3d* d) {
// set origin
m[0] = 0.0;
d[0] = { 0, 0, 0, 1 };
v[0] = { 0, 0, 0, 1 };
float nominal_mass = 2.0e30;
for (int i = 1; i < _num_bodies; i++) {
m[i] = (rand() / (float)RAND_MAX) * nominal_mass + nominal_mass / 2;
double r = (rand() / (double)RAND_MAX) * _scale;
double theta = (rand() / (double)RAND_MAX) * 2 * M_PI;
d[i].x = cos(theta) * r;
d[i].y = sin(theta) * r;
d[i].z = 0.0;
d[i].a = 1.0;
v[i].x = 0.0;
v[i].y = 0.0;
v[i].z = 0.0;
v[i].a = 1.0;
}
}</code></pre>
Notice that <i>r</i> varies between 0 and <span style="font-family: courier;">_scale</span> for the radius, and <i>θ</i> varies between 0 and 2π for the angle, which fills the entire viewable area and then some with stars. Now, let's take another look at the simulation:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="492" src="https://www.youtube.com/embed/rT5XcOhsUhE" width="592" youtube-src-id="rT5XcOhsUhE"></iframe></div><div><br /></div><div>While the initialization now creates a star cluster that looks much more like the real thing, since nothing is moving again, we end up with similar behavior to the first simulation where the stars that start out very close together gravitationally interact first, and they tend to get violently ejected from the system. The difference here is that because the stars in the center are packed much closer together, these interactions happen much more frequently in the center. Most of the outer stars remain fairly stationary for a long time, though, and overall the simulation is only mildly more interesting than the last one. Let's keep going by giving every star a random initial velocity, too:</div>
<pre><code class="cpp">#define M_PI 3.14159265358979323846
void initBodies(float* m, vec3d* v, vec3d* d) {
// set origin
m[0] = 0.0;
d[0] = { 0, 0, 0, 1 };
v[0] = { 0, 0, 0, 1 };
float nominal_mass = 2.0e30;
for (int i = 1; i < _num_bodies; i++) {
m[i] = (rand() / (float)RAND_MAX) * nominal_mass + nominal_mass / 2;
double r = (rand() / (double)RAND_MAX) * _scale;
double theta = (rand() / (double)RAND_MAX) * 2 * M_PI;
d[i].x = cos(theta) * r;
d[i].y = sin(theta) * r;
d[i].z = 0.0;
d[i].a = 1.0;
v[i].x = (rand() / (float)RAND_MAX - 0.5) * 10000.0;
v[i].y = (rand() / (float)RAND_MAX - 0.5) * 10000.0;
v[i].z = 0.0;
v[i].a = 1.0;
}
}</code></pre>
<div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="492" src="https://www.youtube.com/embed/Yec1lP7U4Zw" width="592" youtube-src-id="Yec1lP7U4Zw"></iframe></div><div><br /></div>Now we get a simulation that looks a lot more like the second one, but with a higher concentration of stars in the center. It looks like the whole system will eventually disassociate along with some random violent ejections taking place. This may be a somewhat realistic simulation of an open star cluster, which will eventually disassociate over time like this. However, open star clusters normally only have hundreds of stars, not thousands. A cluster of thousands of stars is most likely a globular cluster that exists for billions of years and doesn't readily disassociate.<div><br /></div><div>We can try another modification to give the stars an initial common motion, and see what happens. Here we'll want to give them a roughly circular motion that's relative to how much gravity they would experience based on how far they are from the center of the cluster. Stars closer to the center should logically be orbiting faster than those stars on the outskirts of the cluster. We can create this kind of motion by setting each star's initial velocity with a magnitude (<i>r</i>) proportional to the square root of the total mass and inverse square root of the star's distance from the center, and an angle that's perpendicular to the star's position angle from the x-axis. Polar coordinates makes this calculation very simple:<br />
<pre><code class="cpp">#define M_PI 3.14159265358979323846
void initBodies(float* m, vec3d* v, vec3d* d) {
// set origin
m[0] = 0.0;
d[0] = { 0, 0, 0, 1 };
v[0] = { 0, 0, 0, 1 };
float nominal_mass = 2.0e30;
double total_mass = m[0];
for (int i = 1; i < _num_bodies; i++) {
m[i] = (rand() / (float)RAND_MAX) * nominal_mass + nominal_mass / 2;
total_mass += m[i];
}
for (int i = 1; i < _num_bodies; i++) {
double r = (rand() / (double)RAND_MAX) * _scale;
if (r == 0.0) r = 1.0e15;
double theta = (rand() / (double)RAND_MAX) * 2 * M_PI;
d[i].x = cos(theta) * r;
d[i].y = sin(theta) * r;
d[i].z = 0.0;
d[i].a = 1.0;
r = sqrtf(G * total_mass / r) * (rand() / (double)RAND_MAX + 3.0) / 4.0;
theta += M_PI / 2 + (rand() / (double)RAND_MAX - 0.5) * M_PI / 2;
v[i].x = cos(theta) * r;
v[i].y = sin(theta) * r;
v[i].z = 0.0;
v[i].a = 1.0;
}
}</code></pre>
We needed to break up the for loop so that all masses are assigned first, and we can accumulate a total mass of the system for later use in the calculation of the magnitude of the velocity. We need to take a precaution now for the initial radius of each star from the origin because it cannot be zero, otherwise we'll divide by zero and the star cluster will implode! (Or the simulation just crashes.) I also added a random component to the velocity magnitude and angle: ±25% for the magnitude and ±π/4 for the angle. Our simulation now has a rotational component:<br /></div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="492" src="https://www.youtube.com/embed/PBV_ghT3C-M" width="592" youtube-src-id="PBV_ghT3C-M"></iframe></div><br /><div>This is starting to look very cool! It still looks like the cluster will disassociate over time, starting from the center, but it has much more dynamic behavior. It's starting to look like a reasonable simulation of a star cluster. What we probably need to hold it together is a huge central mass to act as an anchor for the other stars' orbits. We can do that by simply adding a million solar mass object at the origin (this would obviously be a black hole):</div>
<pre><code class="cpp"> m[0] = 2.0e36;</code></pre>
For this simulation, I also removed the random components to the initial velocities, just to see what it would look like:<br /><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="492" src="https://www.youtube.com/embed/Fvp2LcDvZAk" width="592" youtube-src-id="Fvp2LcDvZAk"></iframe></div><br /><div>This is another interesting simulation that seems to model what a very small galaxy would look like, or possibly a large star cluster. I'm not sure, since I'm not an expert in the subject, but it's fascinating to watch, nonetheless. At times it even looks like spiral arms are forming in rotational waves of stars, although they aren't maintained. It would probably require many more stars to create stable spiral arms. We also still see ejections in this simulation, which would be considered violent relaxations of the system, as one body is ejected and the remaining body in the interaction probably becomes more tightly coupled to the rest of the system. It also looks like many of these ejections happen because of interactions with the central mass that cause a star to whip around the center, accelerate rapidly, and fly out of the system. Let's take one final look at another simulation with the random component added back into the initial velocities:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="492" src="https://www.youtube.com/embed/0_17TdQ0rpc" width="592" youtube-src-id="0_17TdQ0rpc"></iframe></div><br /><div>Wow! Now this is an impressive simulation. It looks like there are multiple layers of orbital motion from the added randomness of the initial velocities, and the system has a pleasingly complex structure to it. There are violent relaxations going on, stars that seem to be falling into the center, and other stars migrating to the outskirts of the cluster. Over time it seems to be fairly stable as well, and the simulation can run for a very long time without losing its general form. Overall, I'm very happy with the results.</div><div><br /></div><div>Like all programming projects, there are plenty of things that could be improved upon in this simulator. We could, of course, implement a model of collisions that would combine stars that came too close together. We could more accurately model acceleration and velocity, as some of those ejections seem to be happening at much higher speeds than they should. Adding in a Lorentz transformation for the velocity calculation would make that more accurate. And, we could strive to improve the simulation engine to be able to simulate a much larger number of bodies. Adding in calculations that keep an ordering of bodies and preferentially calculate forces only for bodies that are closer to each other or extremely massive could potentially allow the simulator to handle orders of magnitude more bodies at the expense of making the kernel much more complex and somewhat less accurate.</div><div><br /></div><div>All of these options would be interesting paths to explore, but for now, this CUDA gravity simulator is looking pretty good and produces some fascinating simulations. It's worth taking a step back and thinking about the amazingly complex calculations that are going on here. Each and every body in the simulation is interacting with every other body, the motion is only set with the initial conditions and then is fully determined by the single force of gravity, and this is all calculated and displayed in real time by a graphics card with 640 execution cores. It's incredible! I'm quite pleased with where the simulator ended up, and I hope you learned some things about CUDA (and gravity) along the way.</div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-78009670407723363342021-02-15T12:55:00.004-06:002021-03-20T15:26:35.949-05:00Playing with Gravity to Learn CUDA: An N-Body Simulation<p>Now that we have <a href="https://sam-koblenski.blogspot.com/2020/12/playing-with-gravity-to-learn-cuda-2.html">a working simulation engine</a> and <a href="https://sam-koblenski.blogspot.com/2021/01/playing-with-gravity-to-learn-cuda.html">a real-time display of a running simulation</a>, it's time to see what we can do with this gravity simulator we've been building in CUDA. We'll start off with a complete simulation of the solar system to see if we can get a reasonable multi-body system simulating correctly. Then we'll move on to filling up the 640 cores on my GeForce GTX 1050 card to make full use of the GPU and see where the base limits are for this simulator. This should be fun.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEju383Vs9StqLKetA-bkiCK8LE4Xa69O8spDaFCDxvqC3t8VOhadrp4-KFg1W6IVaDj2r0EqlHXs7D1Nl7kxSyt_EI6wFto_iTnJ0o9aAxWvQZAulmVYKNCsoCyil3ZV5O2DRJEfe0hN7s/s1500/solar_system.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Model of the Solar System" border="0" data-original-height="1019" data-original-width="1500" height="271" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEju383Vs9StqLKetA-bkiCK8LE4Xa69O8spDaFCDxvqC3t8VOhadrp4-KFg1W6IVaDj2r0EqlHXs7D1Nl7kxSyt_EI6wFto_iTnJ0o9aAxWvQZAulmVYKNCsoCyil3ZV5O2DRJEfe0hN7s/w400-h271/solar_system.jpg" width="400" /></a></div><br /><a name='more'></a><h4 style="text-align: left;">The Solar System</h4><script>hljs.initHighlightingOnLoad();</script><div>Before we're able to run a simulation of the whole solar system, we need to know where all of the planets are at a specific time and what their velocity vectors are. Just knowing their average distance from the Sun and their average speed isn't going to cut it in this case because the planetary orbits are not exactly circular and each planet's mass will influence the others. If we start them all off on the x axis for the initial conditions, the simulation will be something, but it won't be as accurate of a depiction of the real solar system as it could be. </div><div><br /></div><div>For this data Wikipedia is not enough, so instead I turned to the <a href="https://ssd.jpl.nasa.gov/horizons.cgi#top" target="_blank">JPL HORIZONS ephemeris</a> website. Ephemeris is a fancy word for a table of orbital data for objects in the solar system. JPL keeps track of over a million different asteroids, comets, and the planets, which is just incredible to think about, but we're only concerned with ten of those objects. I grabbed the X-Y positions and velocities for the Sun, the planets, and Pluto all at the same time and day in km/s. I intentionally ignored the Z-axis data since we're not using that in the simulator. All of the bodies are assumed to be in the same plane, which isn't entirely accurate but should be close enough, even for Pluto because it's mass is so much smaller than the nearest planet, Neptune, and so it won't have much effect on the rest of the system. I also got the masses of each body from the <a href="https://ssd.jpl.nasa.gov/?planet_phys_par" target="_blank">JPL planet physical characteristics table page</a>.</div><div><br /></div>Since the Sun starts at the origin in this configuration of the data, I needed to figure out the momentum of the rest of the system so that I could set the Sun's velocity to compensate for it. Otherwise, the system would slowly drift in the direction of that momentum. I put the velocity and mass data in a spreadsheet and calculated the velocity that the Sun should have with the following equation:<div><br /></div><div><b>v</b><sub>Sun</sub> = -(Σm<sub>i</sub><b>v</b><sub>i</sub>) / m<sub>Sun</sub></div><div><br /></div><div>Where <b>v</b> stands for the velocity vector of each object and m stands for mass. With the compensating velocity of the Sun calculated, we can plug in the masses, positions, and velocities of each body in our arrays for initial conditions in <span style="font-family: courier;">main()</span>, being careful to convert km to meters for the positions and velocities:<br />
<pre><code class="cpp">int main(int argc, char** argv) {
initGL(&argc, argv);
const float m[_num_bodies] = {
0.0, // Origin
1.989e30, // Sol
3.302e23, // Mercury
4.8685e24, // Venus
5.9724e24, // Earth
6.4171e23, // Mars
1.8981e27, // Jupiter
5.6832e26, // Saturn
8.6813e25, // Uranus
1.0241e26, // Neptune
1.307e22 // Pluto
};
const vec3d d0[_num_bodies] = {
{0, 0, 0, 1}, // Origin
{1, 0, 0, 1}, // Sol
{3.661980629929986E+10, 3.055470346266923E+10, 0, 1}, // Mercury
{-5.873169609724162E+9, -1.085620928662236E+11, 0, 1}, // Venus
{-7.999642430933115E+10, 1.236118155911581E+11, 0, 1}, // Earth
{5.102717483955609E+10, 2.243131860385265E+11, 0, 1}, // Mars
{4.743960342783579E+11, -5.952396785448154E+11, 0, 1}, // Jupiter
{8.356241104235727E+11, -1.237809014804413E+12, 0, 1}, // Saturn
{2.288075301350948E+12, 1.873673832632885E+12, 0, 1}, // Uranus
{4.408978859574347E+12, -7.724145243644576E+11, 0, 1}, // Neptune
{2.113117481871498E+12, -4.659161032056704E+12, 0, 1} // Pluto
};
const vec3d v0[_num_bodies] = {
{0, 0, 0, 1}, // Origin
{1.159902211052130E+01, 1.035640015302160E+01, 0, 1}, // Sol
{-4.080026420194861E+4, 3.948947880398944E+4, 0, 1}, // Mercury
{3.473509242264790E+4, -2.025387845405878E+3, 0, 1}, // Venus
{-2.548479662572985E+4, -1.630237154083575E+4, 0, 1}, // Earth
{-2.270904598757260E+4, 7.431844827548277E+3, 0, 1}, // Mars
{1.007104907684151E+4, 8.767860631883241E+3, 0, 1}, // Jupiter
{7.478797389642545E+3, 5.389266802048683E+3, 0, 1}, // Saturn
{-4.355463670767629E+3, 4.959991511134584E+3, 0, 1}, // Uranus
{9.122491384080702E+2, 5.395758658250224E+3, 0, 1}, // Neptune
{5.072923308461469E+3, 1.070490921773572E+3, 0, 1} // Pluto
};
init(m, v0, d0);
glutDisplayFunc(display);
glutReshapeFunc(reshape);
glutIdleFunc(idle);
glutMainLoop();
finalize();
return 0;
}</code></pre><div>Notice that the position for the Sun was offset by 1 meter in the x direction, otherwise we would have a zero-distance situation between the Origin and the Sun, which would make the Sun disappear in the calculations. Now we need to adjust the scale and timestep for viewing the whole solar system because it's currently set up for a field of view that extends out to the Earth. Setting the <span style="font-family: courier;">_scale</span> to 9.0e12 and <span style="font-family: courier;">_dt</span> to 9000 worked fairly well for seeing the whole solar system:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="484" src="https://www.youtube.com/embed/mk5MW_PF88A" width="583" youtube-src-id="mk5MW_PF88A"></iframe></div><br /><div>Clearly, the inner planets move much faster than the outer planets, and the timestep was set high enough to see some motion in the outer planets without having to wait forever. Meanwhile, the inner planets are whipping around the Sun many times in this one and a half minute video. If we instead want to take a closer look at the inner planets out to Mars, we can set <span style="font-family: courier;">_scale</span> to 9.0e11 and <span style="font-family: courier;">_dt</span> to 1800 for a time step of 30 minutes.</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="491" src="https://www.youtube.com/embed/cTkjJz2ukyQ" width="591" youtube-src-id="cTkjJz2ukyQ"></iframe></div><br /><div>With this view, we can see the four inner planets clearly, and Jupiter makes an appearance on the right side of the video in the beginning and then reappears in the upper left corner near the end of the video. It appears that the simulation is working the way it's supposed to for the solar system, which is promising and exciting.</div><div><br /></div><h4 style="text-align: left;">Scaling Up N</h4><div>Now that we have a working simulation with 10 bodies, let's try scaling that way up. We're not going to try to recreate the asteroid belt or anything in this case. That would be way too tedious. Instead, let's take a different tack and generate bodies with random mass and position. We'll start them all off with zero velocity for this attempt. We can add a function for initializing these bodies as follows:</div>
<pre><code class="cpp">void initBodies(float* m, vec3d* v, vec3d* d) {
// set origin
m[0] = 0.0;
d[0] = { 0, 0, 0, 1 };
v[0] = { 0, 0, 0, 1 };
m[1] = _scale * _scale * 100;
d[1] = { 1, 0, 0, 1 };
v[1] = { 0, 0, 0, 1 };
for (int i = 2; i < _num_bodies; i++) {
m[i] = rand() * _scale / 2 + _scale / 2;
m[i] *= m[i];
d[i].x = (rand() / (float)RAND_MAX - 0.5) * _scale * 2;
d[i].y = (rand() / (float)RAND_MAX - 0.5) * _scale * 2;
d[i].z = 0.0;
d[i].a = 1.0;
v[i].x = 0.0;
v[i].y = 0.0;
v[i].z = 0.0;
v[i].a = 1.0;
}
}</code></pre>The function takes arguments of arrays for the mass, velocity, and position, and it's going to fill these arrays with the random initial conditions for each of the bodies. I try to set a zero-mass origin and a large central mass that's slightly offset from it, but it turns out that there's enough other mass in the configuration that the central mass doesn't act as the intended anchor and instead slowly flies off, taking the origin point with it. Heh, heh. Oh, well, it was worth a try. You can see how the for loop creates random bodies with masses uniformly in the range of [<span style="font-family: courier;">_scale</span>/2, <span style="font-family: courier;">_scale</span>] and positions uniformly distributed around the viewable area established by <span style="font-family: courier;">_scale</span>. I found this to create a relatively interesting simulation for viewing. The arrays can be initialized from <span style="font-family: courier;">main()</span> by calling <span style="font-family: courier;">initBodies()</span> like so:<br />
<pre><code class="cpp">int main(int argc, char** argv) {
initGL(&argc, argv);
float* m = new float[_num_bodies];
vec3d* d0 = new vec3d[_num_bodies];
vec3d* v0 = new vec3d[_num_bodies];
initBodies(m, v0, d0);
init(m, v0, d0);
glutDisplayFunc(display);
glutReshapeFunc(reshape);
glutIdleFunc(idle);
glutMainLoop();
finalize();
return 0;
}
</code></pre></div>With this setup, 1024 bodies, the scale set to 9.0e12, and the timestep set at 900 we get a simulation like the following:<div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="489" src="https://www.youtube.com/embed/lH_pHA9flkA" width="587" youtube-src-id="lH_pHA9flkA"></iframe></div><br /><div>First, it should be noted that the 1024 bodies is more than the number of cores in my GTX 1050 card, but CUDA takes care of allocating the threads to cores automatically. The maximum number of threads in a block is 1024, and we're currently using a single block, so this is the maximum number of bodies for the simulation at the moment. It looks like the card can handle this number easily.</div><div><br /></div><div>The other thing that should be immediately obvious from this simulation is that most of the bodies are ejected from the scene relatively quickly. That may seem surprising because we may expect with a starting velocity of zero, all of that mass would gravitate towards the center, but we should take some other things into consideration here. First, the initial setup is not at all like anything we see in space. We have an area about the size of our solar system that's uniformly filled with 1,024 objects that are about the mass of large asteroids, and none of them are moving. As soon as one of these bodies has a close interaction with another body, it is quite likely to accelerate significantly and get thrown out of the view and the system.</div><div><br /></div><div>If we wanted to try to simulate a proto-planetary system, we would need to start with a ridiculously large amount of bodies that were on the scale of dust particles. That type of simulation at that scale isn't currently possible with this setup, and it might be impossible with a GPU of this performance level. If we wanted to sidestep that issue and start at a later time during the system's formation when larger bodies have already started to form, we would need to have the bodies moving in a generally circular direction around the center, and at least some of them would likely need to be much more massive. That simulation might be more possible with this GPU, but the initial conditions would need to be set with more intention to get a reasonable result. Also, the simulation engine has no way to combine bodies when collisions occur. It doesn't even detect collisions or have a sense of the physical size of the bodies, so that type of simulation wouldn't really work. Plus, the timescales we're talking about with these types of simulations would be millions of years, and the current timestep was half an hour. That would need some adjustment.</div><div><br /></div><div>Another possible option for a more realistic simulation is trying to simulate a galaxy. In order to do that, we would have to scale way out in viewing area, mass of bodies, number of bodies, and timestep. Collisions are far less likely at the scale of galaxies, so that aspect could be safely ignored, but galaxies have hundreds of millions or billions of stars in them and timescales are at least in millions of years to see reasonable amounts of motion. We may be able to look at something like a star cluster, but a galaxy simulation might be out of the realm of possibility with the hardware I'm working with and the way the current engine is set up.</div><div><br /></div><div>It may be worth looking into setting up a star cluster simulation for the next episode. Star clusters will have thousands to tens of thousands of stars, which is certainly within the realm of possibility for these simulations, and at least open star clusters will disassociate over time, just like this 1024-body simulation did. Having initial conditions of zero velocity or small random velocities may be a reasonable starting point for such a simulation, so that'll be the plan while we see what we can do about optimizing the simulation to enable a much higher number of bodies at once.</div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0Madison, WI, USA43.0730517 -89.401230214.762817863821155 -124.5574802 71.383285536178846 -54.2449802tag:blogger.com,1999:blog-6227791252039855587.post-74771924196532580952021-01-25T19:47:00.000-06:002021-01-25T19:47:52.646-06:00Playing with Gravity to Learn CUDA: Simulation Display<p>We're building a gravity simulation on a GPU using CUDA, and last time we actually got a <a href="https://sam-koblenski.blogspot.com/2020/12/playing-with-gravity-to-learn-cuda-2.html">working simulator running for a 2-body system</a>. That was exciting, but there were some drawbacks, namely the printout-copy-paste-to-spreadsheet-and-graph way of visualizing the simulation. In this post we're going to remedy that issue with a simulation display that shows the position of the bodies graphically while the simulation is running and that also runs on the graphics card alongside the simulation. We'll even make the position buffer multipurpose so that we can calculate the positions directly into this buffer, and then turn around and draw those positions into a window from the same buffer. No copying required, not even automated copying. </p><p>Making this display for the gravity simulator turned out to be more difficult than I thought because I haven't programmed in OpenGL since the graphics course I took in college, and I've certainly never done any OpenGL-CUDA interop before. I managed to pull something together by leaning heavily on the N-Body sample project that's part of the nVidia CUDA sample projects. This sample project is also a gravity simulator, but the underlying simulation engine is substantially different than the one we've built so far. Even so, I was able to use the renderer without any modifications. Let's see how it works.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_1ahgNB4hQ8sk91NNz7yWwmf3d0qNFpANniZR661OMU6zv36syslkEmxtEdU-hQC5691R9FqWzuewsm2X4t4XksBYoTu3ITt6h9lGYsBCa8p84ozRF-dDtwy6FtAAoEvc92woc7Op8dI/s1024/earth_sun_orbit2.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Earth in the Sun's orbit" border="0" data-original-height="573" data-original-width="1024" height="224" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_1ahgNB4hQ8sk91NNz7yWwmf3d0qNFpANniZR661OMU6zv36syslkEmxtEdU-hQC5691R9FqWzuewsm2X4t4XksBYoTu3ITt6h9lGYsBCa8p84ozRF-dDtwy6FtAAoEvc92woc7Op8dI/w400-h224/earth_sun_orbit2.jpg" width="400" /></a></div><br /><a name='more'></a><h4 style="text-align: left;">Initialization</h4><script>hljs.initHighlightingOnLoad();</script><div>When discussing the simulation kernel, we took a bottom-up approach, but now that the kernel already exists and won't change much while we add the display, we'll take a top-down approach for the changes that go into building the simulation display. The first thing we'll do is move <span style="font-family: courier;">main()</span> to a new file called <span style="font-family: courier;">multi_body_gravity.cpp</span>, since it will no longer only set up the simulation engine. This file will contain both OpenGL and CUDA code, so it's going to need a few more headers and some global objects and constants:</div>
<pre><code class="cpp">#include <helper_gl.h>
#if defined(WIN32) || defined(_WIN32) || defined(WIN64) || defined(_WIN64)
#include <GL/wglew.h>
#endif
#include <GL/freeglut.h>
#include <cuda_runtime.h>
#include <cuda_gl_interop.h>
#include <helper_cuda.h>
#include "render_particles.h"
#include "kernel.h"
ParticleRenderer* _renderer;
unsigned int _pbo[2];
cudaGraphicsResource* _pGRes[2];
const float _scale = 3.0e11;
const float _point_size = _scale / 50.0;
const int _num_bodies = 2;
const float _dt = 900;</code></pre>Some of these includes are found directly in the CUDA samples project directory, so I had to add references to those subdirectories in the project properties for the C/C++ and CUDA compiler include directories and linker library directories. The <span style="font-family: courier;">ParticleRenderer</span> class is actually the unmodified renderer from the N-Body sample project, so I copied the <span style="font-family: courier;">render_particles.h</span> and <span style="font-family: courier;">render_particles.cpp</span> files over to my project. The <span style="font-family: courier;">_pbo[2]</span> array is the pair of pixel buffer objects that will hold the positions of each body in the simulation. The <span style="font-family: courier;">_pGRes[2]</span> array is the corresponding CUDA graphics resources that enable the connection between CUDA and OpenGL through those pixel buffer objects. We'll see the initialization code that sets these structures up in a minute. Finally, we have a few global simulation constants. The <span style="font-family: courier;">_scale</span> and <span style="font-family: courier;">_point_size</span> make it easy to set the area of the simulation that the camera will see and display so that the bodies are actually visible in the display window. The <span style="font-family: courier;">_num_bodies</span> and <span style="font-family: courier;">_dt</span> were promoted to global constants from the initialization code of the simulator because they'll be used more extensively throughout the initialization code.<div><br /></div><div>Now let's take a look at <span style="font-family: courier;">main()</span> because it's changed quite a bit:</div>
<pre><code class="cpp">int main(int argc, char** argv) {
initGL(&argc, argv);
const float m[_num_bodies] = { 1.989e30, 5.972e24 };
const vec3d d0[_num_bodies] = { {0, 0, 0, 1}, {1.496e11, 0, 0, 1} };
const vec3d v0[_num_bodies] = { {0, -8.9415e-2, 0, 1}, {0, 2.978e4, 0, 1} };
init(m, v0, d0);
glutDisplayFunc(display);
glutReshapeFunc(reshape);
glutIdleFunc(idle);
glutMainLoop();
finalize();
return 0;
}</code></pre>The calls to the simulation engine are gone, and a bunch of other initialization calls are here instead. We start by calling <span style="font-family: courier;">initGL()</span> to initialize the OpenGL system, and we pass in the command line arguments because we can pass OpenGL command line configuration values into the OpenGL subsystem this way. Then we initialize the mass, position, and velocity of the bodies like we did before (I will likely make this its own function in the future, which is why it's still sitting in <span style="font-family: courier;">main()</span> for now), but notice that we're using <span style="font-family: courier;">vec3d</span> instead of <span style="font-family: courier;">vec2d</span> structs. I had to switch to using a 3D vector with alpha, since that's the point format that the renderer expects. This struct is now in a new header <span style="font-family: courier;">kernel.h</span>:
<pre><code class="cpp">typedef struct vec3d {
float x;
float y;
float z;
float a;
} vec3d;
cudaError_t initCuda(const float* m, const vec3d* v0, unsigned int size);
cudaError_t gravitySim(vec3d** d, const float dt, unsigned int size);
void finalizeCuda();</code></pre>As you can see, there are a couple of new functions in the kernel, too, but we'll get to those later. For now, back in <span style="font-family: courier;">main()</span> we call <span style="font-family: courier;">init()</span> to initialize the rest of the simulator and renderer. Then, we define the OpenGL display, reshape, and idle functions before calling the main loop that starts drawing frames to the screen. This loop is where <span style="font-family: courier;">display()</span> gets called over and over again. Finally, we call <span style="font-family: courier;">finalize()</span> to close up shop when the program closes. We'll take a look at each of these functions in turn, starting with <span style="font-family: courier;">initGL()</span>:
<pre><code class="cpp">void initGL(int* argc, char** argv) {
// First initialize OpenGL context, so we can properly set the GL for CUDA.
// This is necessary in order to achieve optimal performance with OpenGL/CUDA interop.
glutInit(argc, argv);
glutInitDisplayMode(GLUT_RGB | GLUT_DEPTH | GLUT_DOUBLE);
glutInitWindowSize(720, 480);
glutCreateWindow("CUDA n-body gravity simulator");
if (!isGLVersionSupported(2, 0) ||
!areGLExtensionsSupported(
"GL_ARB_multitexture "
"GL_ARB_vertex_buffer_object")) {
fprintf(stderr, "Required OpenGL extensions missing.");
exit(EXIT_FAILURE);
} else {
#if defined(WIN32)
wglSwapIntervalEXT(0);
#elif defined(LINUX)
glxSwapIntervalSGI(0);
#endif
}
glEnable(GL_DEPTH_TEST);
glClearColor(0.0, 0.0, 0.0, 1.0);
checkGLErrors("initGL");
}</code></pre>The first few API calls here are fairly self-explanatory. Then, there's the if-else statement that is all required for the vertex and pixel shaders that are used in the renderer. (I tried calling <span style="font-family: courier;">wglSwapIntervalEXT()</span> immediately since I know I'm not on a Linux system, but that did not work.) Before returning from the function, we enable the GL_DEPTH_TEST to enable z-clipping, clear the window color to black, and check for errors. Next up in <span style="font-family: courier;">main()</span> is the <span style="font-family: courier;">init()</span> function:
<pre><code class="cpp">void init(const float* m, const vec3d* v0, const vec3d* d0) {
// create the position pixel buffer objects for rendering
// we will actually compute directly from this memory in CUDA too
glGenBuffers(2, (GLuint*)_pbo);
unsigned int memSize = sizeof(float) * 4 * _num_bodies;
for (int i = 0; i < 2; ++i) {
glBindBuffer(GL_ARRAY_BUFFER, _pbo[i]);
glBufferData(GL_ARRAY_BUFFER, memSize, d0, GL_DYNAMIC_DRAW);
int size = 0;
glGetBufferParameteriv(GL_ARRAY_BUFFER, GL_BUFFER_SIZE, (GLint*)&size);
if ((unsigned)size != memSize) {
fprintf(stderr, "WARNING: Pixel Buffer Object allocation failed!n");
}
glBindBuffer(GL_ARRAY_BUFFER, 0);
checkCudaErrors(cudaGraphicsGLRegisterBuffer(&_pGRes[i], _pbo[i], cudaGraphicsMapFlagsNone));
}
initCuda(m, v0, _num_bodies);
_renderer = new ParticleRenderer;
float color[4] = { 1.0f, 0.6f, 0.3f, 1.0f };
_renderer->setBaseColor(color);
float* hColor = new float[_num_bodies * 4];
for (int i = 0; i < sizeof(hColor); i += 4) {
hColor[i] = color[0];
hColor[i + 1] = color[1];
hColor[i + 2] = color[2];
hColor[i + 3] = color[3];
}
_renderer->setColors(hColor, _num_bodies);
_renderer->setSpriteSize(_point_size);
_renderer->setPBO(_pbo[1], _num_bodies, false);
}</code></pre>This part of the initialization starts off by generating the two pixel buffer objects, and loading them with the initial position data using <span style="font-family: courier;">glBufferData()</span>. Next, we need to allocate the CUDA device buffers, but this no longer includes the position buffers, so that initialization code is somewhat reduced from before:
<pre><code class="cpp">cudaError_t initCuda(const float* m, const vec3d* v0, unsigned int size)
{
cudaError_t cudaStatus;
// Choose which GPU to run on, change this on a multi-GPU system.
cudaStatus = cudaSetDevice(0);
// Allocate GPU buffers for 3 vectors (two input, one output)
cudaStatus = cudaMalloc((void**)&dev_v, size * sizeof(vec3d));
cudaStatus = cudaMalloc((void**)&dev_m, size * sizeof(float));
cudaStatus = cudaMalloc((void**)&dev_v0, size * sizeof(vec3d));
// Copy input vectors from host memory to GPU buffers.
cudaStatus = cudaMemcpy(dev_m, m, size * sizeof(float), cudaMemcpyHostToDevice);
cudaStatus = cudaMemcpy(dev_v0, v0, size * sizeof(vec3d), cudaMemcpyHostToDevice);
return cudaStatus;
}</code></pre>Back in <span style="font-family: courier;">init()</span> we then configure the renderer with its base color, colors for each body, the sprite size of the bodies, and the pixel buffer object. Notice that we only set the second of the two PBOs because we're going to take a shortcut and calculate two timesteps with each simulation pass. This trick allows us to make the simulation twice as accurate because we can reduce the timestep, interchange the buffers automatically with two function calls, and always use the same PBO so we can set it in initialization and forget it. Technically, this means we only need one PBO, but this setup works just fine. That's it for initialization, so let's look at how we display the points in the window.<div><br /></div><h4 style="text-align: left;">Display</h4><div>This is where the magic happens. Each time the window is redrawn, OpenGL calls <span style="font-family: courier;">display()</span> to do the drawing to the screen. We need to step the simulation during this time to know where to draw the bodies next, and this is what it looks like:</div>
<pre><code class="cpp">void display() {
vec3d* dev_d[2];
checkCudaErrors(cudaGraphicsMapResources(2, _pGRes, 0));
size_t bytes;
checkCudaErrors(cudaGraphicsResourceGetMappedPointer((void**)&(dev_d[0]), &bytes, _pGRes[0]));
checkCudaErrors(cudaGraphicsResourceGetMappedPointer((void**)&(dev_d[1]), &bytes, _pGRes[1]));
gravitySim(dev_d, _dt, _num_bodies);
checkCudaErrors(cudaGraphicsUnmapResources(2, _pGRes, 0));
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glTranslatef(0, 0, -_scale);
_renderer->display(ParticleRenderer::PARTICLE_SPRITES_COLOR);
glutSwapBuffers();
glutReportErrors();
}</code></pre>In the first half of this <span style="font-family: courier;">display()</span> function, we are mapping the graphics resource that we had set up in the initialization to a pair of <span style="font-family: courier;">vec3d</span> pointers. These pointers to buffers are what the <span style="font-family: courier;">dev_d0</span> and <span style="font-family: courier;">dev_d</span> buffers used to be, and we can pass them into the gravity simulator to use in the kernel as if they are just like any other GPU memory buffer that was allocated with CUDA. On the first call to <span style="font-family: courier;">display()</span>, these buffers will contain the initial positions that were copied in for the bodies, and after that they'll contain the calculated positions after each time step. The rest of the <span style="font-family: courier;">display()</span> function clears the window, sets up the camera view to point at the origin from <span style="font-family: courier;">-_scale</span> distance in the z-direction, tells the renderer to draw the bodies using the pixel buffer objects, and finally, swap the ping-pong screen buffers and report any OpenGL errors. <div><br /></div><div>Because we're still calculating positions and velocities with 2D points, even though they're inside 3D points, all of the bodies will appear on the x-y plane, and the camera will be looking directly at the X-Y plane from far away on the z-axis. We won't go into how the renderer actually draws the bodies since that <span style="font-family: courier;">_renderer->display()</span> function was used as-is, but we will take a deeper look at that <span style="font-family: courier;">gravitySim()</span> function call:
<pre><code class="cpp">cudaError_t gravitySim(vec3d** dev_d, const float dt, unsigned int size) {
cudaError_t cudaStatus = cudaSuccess;
vec3d *d = new vec3d[size];
cudaStatus = gravityIter(d, dev_v, dev_d[1], dev_m, dev_v0, dev_d[0], dt, size);
cudaStatus = gravityIter(d, dev_v0, dev_d[0], dev_m, dev_v, dev_d[1], dt, size);
return cudaStatus;
}</code></pre>The call to run the gravity simulation has become extremely simple. It's just two calls to <span style="font-family: courier;">gravityIter()</span> to step the simulation twice with <span style="font-family: courier;">dev_d[0]</span> as the input positions and <span style="font-family: courier;">dev_d[1]</span> as the output positions for one call, and then they're swapped for the second call. The <span style="font-family: courier;">dev_v0</span> and <span style="font-family: courier;">dev_v</span> pointers to the velocity buffers are swapped for each call as well. The <span style="font-family: courier;">gravityIter()</span> function is unmodified, so it still copies the positions from the GPU memory back to host memory, requiring us to pass in a <span style="font-family: courier;">vec3d</span> array to hold the values. This feature is nice for debugging, but once things are working that extra GPU memory transfer can be disabled to speed up the simulation. <div><br /></div><div>That's about it for drawing body positions to the screen, but there are two other functions that we set up with OpenGL that get called periodically. The <span style="font-family: courier;">reshape()</span> function will get called whenever the simulation window gets resized, and it looks like this:
<pre><code class="cpp">void reshape(int w, int h) {
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(60.0, (float)w / (float)h, _scale * 2 / 1.0e6, _scale * 2);
glMatrixMode(GL_MODELVIEW);
glViewport(0, 0, w, h);
}</code></pre>All this function does is adjust the camera perspective to match the new window dimensions. The last two arguments in <span style="font-family: courier;">gluPerspective()</span> are for the near and far draw distance, and only the objects within this range will actually be drawn to the window. The <span style="font-family: courier;">_scale</span> constant is used to set the draw area, and this is important because we're working with astronomical distances here. The default draw range isn't enough. When I was first testing this out, I could get bodies drawn to the screen for the small debug system, but nothing would get drawn for the Earth-Sun simulation. It actually took a while to realize that these settings in <span style="font-family: courier;">gluPerspective()</span> were why I wasn't getting anything on the screen for the Earth-Sun simulation. The problem was that everything was getting clipped out because the draw distance was so short! So, remember to set the draw distance correctly if you can't see anything in your view.<div><br /></div><div>The last drawing function is also the easiest:</div>
<pre><code class="cpp">void idle(void) {
glutPostRedisplay();
}</code></pre>The <span style="font-family: courier;">idle()</span> function is called whenever no other processing is happening, and all it does is call <span style="font-family: courier;">glutPostRedisplay()</span> to tell the system that it can start drawing the next frame.<div><br /></div><h4 style="text-align: left;">Cleanup</h4><div>We now have initialization and display complete, so all that's left is the cleanup when the program terminates. The cleanup is pretty simple, and it all happens in <span style="font-family: courier;">finalize()</span>:</div>
<pre><code class="cpp">void finalize() {
checkCudaErrors(cudaGraphicsUnregisterResource(_pGRes[0]));
checkCudaErrors(cudaGraphicsUnregisterResource(_pGRes[1]));
glDeleteBuffers(2, (const GLuint*)_pbo);
finalizeCuda();
}</code></pre>First, we unwind the graphics resources and pixel buffer objects that we had created. Then, we call <span style="font-family: courier;">finalizeCuda()</span>:
<pre><code class="cpp">void finalizeCuda() {
cudaFree(dev_v);
cudaFree(dev_m);
cudaFree(dev_v0);
// cudaDeviceReset must be called before exiting in order for profiling and
// tracing tools such as Nsight and Visual Profiler to show complete traces.
checkCudaErrors(cudaDeviceReset());
}</code></pre>This function simply frees the remaining CUDA buffers from the GPU and does a device reset to make sure the debugging tools have accurate data. That completes the modifications to our simulator to enable a graphical display, and I'm pretty happy with the organization. From a high level it consists of a simulation engine using CUDA, a rendering engine using OpenGL, and a controller that sits between them and runs the initialization, interop, and cleanup. It's all nice and tidy.<div><br /></div><h4 style="text-align: left;">Running a Simulation</h4><div>Now that we have a visual display of our simulation, we can take a look at the Earth-Sun orbit in motion:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="266" src="https://www.youtube.com/embed/oirfbhoJUH0" width="320" youtube-src-id="oirfbhoJUH0"></iframe></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div>In order to get this simulation, I lowered the time step to be every half hour. The old value from the last post of every 6 hours resulted in a simulation that went way too fast. The other benefit of the lower time step is increased simulation accuracy, but even with the increased accuracy I saw that the Earth was slowly spiraling towards the top of the screen. To debug this issue, I zoomed in on the Sun and found that it, too, was slowly moving up as the simulation ran. The reason behind this drift was that the initial velocity of the Sun was set to zero, but the Earth had an initial velocity in the positive y-direction. This setup gave the whole system a small amount of momentum in the positive y-direction, causing the Earth and Sun to slowly drift up. To correct for this momentum, I just needed to give the Sun a small velocity in the negative y-direction, and the value needed to be inversely proportional to the Sun's mass and proportional to the Earth's mass and velocity. The exact value of -0.089415 m/s is already shown in the new initial velocity for the Sun in the initialization code section above. With that adjustment, we can zoom in on the Sun by changing the <span style="font-family: courier;">_scale</span> to 2.0e6 and see what the Sun's motion really looks like:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="266" src="https://www.youtube.com/embed/DJ_68vlz5do" width="320" youtube-src-id="DJ_68vlz5do"></iframe></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div>I added a zero-mass reference point at the origin so that it's clear that the Sun returns to its starting point after completing a full circle. This is all very cool, and it now looks like we have a fully functional gravity simulator with a real time display. Of course, I can imagine a number of improvements to the display that would make things even better: dynamic zoom and pan controls, different colors for the different bodies, a grid with distance values, etc. That's all doable stuff, but I want to make more progress on the simulation engine itself, so next time we'll see about scaling up the simulation with more bodies and see how far we can go with this CUDA kernel.</div></div></div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-86081159796289098882020-12-28T19:11:00.000-06:002020-12-28T19:11:53.443-06:00Playing with Gravity to Learn CUDA: A 2-Body Simulation<p>In the last post of this series on learning CUDA through building a gravity simulation, we didn't actually do any CUDA. We <a href="https://sam-koblenski.blogspot.com/2020/12/playing-with-gravity-to-learn-cuda.html">focused on defining gravity</a> and figuring out how we were going to actually simulate it with practical equations. Now it's time to put those equations to work and see if we can come up with a functioning simulation that uses CUDA. We'll start with the simple 2-body problem of the Earth orbiting around the Sun, and see if we can keep the Earth in orbit.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgka1Vo7ar4wN7qr79t-X1zAjSlbY0hPg9bIPXZ1c0G0Lhov5HzrOnDXYrGfUzkVdwcHgXiH6RIgi2XsWZe_8j8l4JtWd_hCgllrkTBIs2UZUX5RYCn3cjy7ObAvQUCshUXFA53quLZnK4/s1200/earth_sun_orbit.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Earth in orbit around the Sun" border="0" data-original-height="675" data-original-width="1200" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgka1Vo7ar4wN7qr79t-X1zAjSlbY0hPg9bIPXZ1c0G0Lhov5HzrOnDXYrGfUzkVdwcHgXiH6RIgi2XsWZe_8j8l4JtWd_hCgllrkTBIs2UZUX5RYCn3cjy7ObAvQUCshUXFA53quLZnK4/w400-h225/earth_sun_orbit.jpg" width="400" /></a></div><div style="text-align: left;"><br /></div><span><a name='more'></a></span><h4 style="text-align: left;">The Kernel</h4><script>hljs.initHighlightingOnLoad();</script><div>Remember, the part of the code that executes on the GPU is the kernel. We're going to start with designing the kernel because I want to get the gravity equations down in code in order to get a handle on which vectors we're going to need to supply to the kernel from the host CPU. We'll build up the rest of the simulation from that kernel, and then we can see how it works. </div><div><br /></div><div>First, we need to think about how this kernel is going to be used during execution. Each thread running on the GPU will execute its own instance of the kernel. We will have <i>n</i> bodies in this simulation, and each body will need to examine every other body to calculate their contribution to the total gravitational force felt by the first body. Said another way, if we have bodies b<sub>1</sub>, ..., b<sub>n</sub>, then we calculate the total force on body b<sub>1</sub> from the contributions of the forces from bodies b<sub>2</sub>, ..., b<sub>n</sub>. It may seem from this O(n<sup>2</sup>) problem that we could gain something by parallelizing every individual body-body calculation, but if <i>n</i> is on the order of the number of cores in the GPU, then we've really gained nothing but difficulty in designing the calculations in the kernel because we're going to have to do <i>n</i> iterations anyway. The iterations are just no longer in a simple inner loop in the thread of one body calculation. We may even suffer a slowdown from too much extra control code to manage everything. It'll be simpler and likely sufficient to have the kernel do the whole calculation for one body, and the simulation will be O(n) as long as <i>n</i> is on the order of the number of cores in the GPU.</div><div><br /></div><div>With that consideration out of the way, we'll start with the gravitational constant, and a simple data structure to hold a 2D point:</div>
<pre><code class="cpp">#include "cuda_runtime.h"
#include "device_launch_parameters.h"
#include <stdio.h>
#define G 6.674e-11
typedef struct vec2d {
float x;
float y;
} vec2d;</code></pre>The includes are the basic CUDA includes that all such programs require, and G is our gravitational constant from the last post. The vec2d struct simply contains x and y elements, and it's intended to be used in vector calculations. It's guaranteed to pack neatly into arrays since there's no extra fluff, and we're going to need that compactness when passing data to and from the GPU.<div><br /></div><div>Next, for the kernel definition we're going to need the mass and current position of each body, the current velocity of the body we're calculating, and we'll output the new position and velocity of that body. All of these variables are passed to the kernel as arrays of data for all bodies because all of the data is loaded into the GPU memory at once, and each thread instance will access the elements that it needs to. The start of the kernel then looks like this:</div>
<pre><code class="cpp">__global__ void gravityKernel(vec2d* v, vec2d* d, const float *m, const vec2d* v0, const vec2d* d0, float dt)
{
int i = threadIdx.x;
const vec2d d0_i = d0[i];
vec2d a = { 0, 0 };</code></pre>The first thing we need to do in the kernel is grab the index for the thread that's currently running. That index <span style="font-family: courier;">threadIdx</span> will allow us to access the correct location in the arrays for the body we're calculating the force on for this thread. We then immediately pull out the current location for that body for easy access during the rest of the calculation. We also initialize a vector for the acceleration for that body, and this vector will accumulate all of the contributing accelerations from the forces from each other body in the simulation. The for loop that follows contains this accumulated acceleration calculation derived from the gravitational force and acceleration formulas from the last post:
<pre><code class="cpp"> for (int j = 0; j < blockDim.x; j++) {
if (j == i) continue;
const vec2d d0_j = d0[j];
vec2d r_ij;
r_ij.x = d0_i.x - d0_j.x;
r_ij.y = d0_i.y - d0_j.y;
float r_squared = r_ij.x * r_ij.x + r_ij.y * r_ij.y;
float F_coef = -G * m[i] * m[j] / r_squared;
vec2d F_ij;
F_ij.x = F_coef * r_ij.x * rsqrtf(r_squared);
F_ij.y = F_coef * r_ij.y * rsqrtf(r_squared);
a.x += F_ij.x / m[i];
a.y += F_ij.y / m[i];
}</code></pre>This code may look like complicated math, but it's a straight translation of the two force equations in vector form:<div><br /><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhheZKLVn8dOqQYTycOmzodIbCwIuw8wcjqsDMZ2kKIn5bkfrA2oXIbgEwbJSPhQfYse60IMdFeZ5GTBJwDieC82I2cOQZR2AZjaWvOZrcwkG6Mjz5hcczJpZdnXfWilyu5pMg5Honr7eU/" style="margin-left: 1em; margin-right: 1em;"><img data-original-height="47" data-original-width="167" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhheZKLVn8dOqQYTycOmzodIbCwIuw8wcjqsDMZ2kKIn5bkfrA2oXIbgEwbJSPhQfYse60IMdFeZ5GTBJwDieC82I2cOQZR2AZjaWvOZrcwkG6Mjz5hcczJpZdnXfWilyu5pMg5Honr7eU/s16000/image.png" /></a></div><div class="separator" style="clear: both; text-align: center;"><b><br /></b></div><div class="separator" style="clear: both; text-align: center;"><b>F</b> = <i>m</i><b>a</b></div></div></div><br /><div>First, we calculate the <b>r<sub>ij</sub></b> vector for the distance from body <i>j</i> to body <i>i</i> (the direction is important), and then just churn through the rest of the equations. The <span style="font-family: courier;">rsqrtf()</span> function calculates <span style="font-family: courier;">1 / sqrtf()</span>, which is faster to do as the reciprocal and then multiply instead of doing the divide by the square root directly. Note that we multiply by <span style="font-family: courier;">m[i]</span> to calculate <span style="font-family: courier;">F_coef</span>, and then we divide by the same <span style="font-family: courier;">m[i]</span> later to calculate the acceleration vector to add, so we can remove those operations and accumulate the acceleration vectors more directly:</div>
<pre><code class="cpp"> for (int j = 0; j < blockDim.x; j++) {
if (j == i) continue;
const vec2d d0_j = d0[j];
vec2d r_ij;
r_ij.x = d0_i.x - d0_j.x;
r_ij.y = d0_i.y - d0_j.y;
float r_squared = r_ij.x * r_ij.x + r_ij.y * r_ij.y;
float F_coef = -G * m[j] / r_squared;
a.x += F_coef * r_ij.x * rsqrtf(r_squared);
a.y += F_coef * r_ij.y * rsqrtf(r_squared);
}
</code></pre>Another thing to note in this code is that <span style="font-family: courier;">blockDim.x</span> is another standard variable from CUDA that gives the size of the block of threads, which in this case translates directly to the number of bodies. It's a convenient variable to use as the loop boundary. Then right at the beginning of each loop instance, we check if the opposing body we're looking at is the same as the one we're calculating the acceleration for, and we skip to the next body in that case. Otherwise, we'll get a divide-by-zero error when we divide by <span style="font-family: courier;">r_squared</span>. The rest of the kernel simply calculates the new velocity and position vectors:
<pre><code class="cpp"> const vec2d v0_i = v0[i];
v[i].x = v0_i.x + a.x * dt;
v[i].y = v0_i.y + a.y * dt;
d[i].x = d0_i.x + v0_i.x * dt + a.x * dt * dt / 2;
d[i].y = d0_i.y + v0_i.y * dt + a.y * dt * dt / 2;
}
</code></pre>There should be no surprises there.<div><br /></div><h4 style="text-align: left;">Calling the Kernel</h4><div>On the next level up in the code, we need to call the kernel from the host CPU. We'll use a helper function to call the kernel and read the newly computed position vector out of the GPU memory:</div>
<pre><code class="cpp">cudaError_t gravityIter(vec2d* d, vec2d* dev_v, vec2d* dev_d, const float* dev_m, const vec2d* dev_v0, const vec2d* dev_d0, float dt, unsigned int size)
{
cudaError_t cudaStatus;
// Launch a kernel on the GPU with one thread for each body.
gravityKernel <<< 1, size >>> (dev_v, dev_d, dev_m, dev_v0, dev_d0, dt);
// Check for any errors launching the kernel
cudaStatus = cudaGetLastError();
if (cudaStatus != cudaSuccess) {
fprintf(stderr, "addKernel launch failed: %s\n", cudaGetErrorString(cudaStatus));
return cudaStatus;
}
// cudaDeviceSynchronize waits for the kernel to finish
cudaStatus = cudaDeviceSynchronize();
// Copy output vector from GPU buffer to host memory.
cudaStatus = cudaMemcpy(d, dev_d, size * sizeof(vec2d), cudaMemcpyDeviceToHost);
return cudaStatus;
}</code></pre>This code differentiates between GPU device memory variables and host CPU memory variables with the "dev_" prefix for GPU variables. That way, we can keep straight where any given variable resides, as that's critically important. Trying to access variables in the wrong memory space will cause a crash. We go ahead and call <span style="font-family: courier;">gravityKernel</span> right away in this function, passing the GPU device variables that it needs and <span style="font-family: courier;">size</span> for the number of threads to create in the special kernel call syntax. Then we check for errors, call a <span style="font-family: courier;">cudaDeviceSynchronize()</span> command to wait for the kernels to finish, and call <span style="font-family: courier;">cudaMemcpy() </span>to copy the position vector on the GPU back to the host CPU memory. The first three arguments should look familiar to anyone that's used <span style="font-family: courier;">memcpy()</span> before, and the final <span style="font-family: courier;">cudaMemcpyDeviceToHost</span> argument specifies which direction we're copying this data.<div><br /></div><div>I showed one example of the error checking code for <span style="font-family: courier;">cudaGetLastError()</span> to give a sense of what basic error checking looks like, but I left out the rest for conciseness. Basically anywhere that <span style="font-family: courier;">cudaStatus</span> is set would have similar error handling. It's useful for debugging if something goes wrong, but uninteresting otherwise. On the next level up, we want to call this <span style="font-family: courier;">gravityIter()</span> function from a loop that will step the gravity simulation. Here's what that looks like:</div>
<pre><code class="cpp">cudaError_t gravitySim(vec2d* d, const float* m, const vec2d* v0, const vec2d* d0, const float dt, unsigned int size, unsigned int iterations)
{
float* dev_m = 0;
vec2d* dev_v0 = 0;
vec2d* dev_d0 = 0;
vec2d* dev_d = 0;
vec2d* dev_v = 0;
cudaError_t cudaStatus;
// Choose which GPU to run on, change this on a multi-GPU system.
cudaStatus = cudaSetDevice(0);
if (cudaStatus != cudaSuccess) {
fprintf(stderr, "cudaSetDevice failed! Do you have a CUDA-capable GPU installed?");
goto Error;
}
// Allocate GPU buffers for five vectors (three input, two output) .
cudaStatus = cudaMalloc((void**)&dev_d, size * sizeof(vec2d));
cudaStatus = cudaMalloc((void**)&dev_v, size * sizeof(vec2d));
cudaStatus = cudaMalloc((void**)&dev_m, size * sizeof(float));
cudaStatus = cudaMalloc((void**)&dev_v0, size * sizeof(vec2d));
cudaStatus = cudaMalloc((void**)&dev_d0, size * sizeof(vec2d));
// Copy input vectors from host memory to GPU buffers.
cudaStatus = cudaMemcpy(dev_m, m, size * sizeof(float), cudaMemcpyHostToDevice);
cudaStatus = cudaMemcpy(dev_v0, v0, size * sizeof(vec2d), cudaMemcpyHostToDevice);
cudaStatus = cudaMemcpy(dev_d0, d0, size * sizeof(vec2d), cudaMemcpyHostToDevice);
int time = 0;
printf("Time,Object0.x,Object0.y,Object1.x,Object1.y\n");
for (int i = 0; i < iterations; i++) {
if (cudaSuccess != (cudaStatus = gravityIter(d, dev_v, dev_d, dev_m, dev_v0, dev_d0, dt, size))) {
goto Error;
}
time += dt;
printf("%d,%f,%f,%f,%f\n", time, d[0].x, d[0].y, d[1].x, d[1].y);
if (cudaSuccess != (cudaStatus = gravityIter(d, dev_v0, dev_d0, dev_m, dev_v, dev_d, dt, size))) {
goto Error;
}
time += dt;
printf("%d,%f,%f,%f,%f\n", time, d[0].x, d[0].y, d[1].x, d[1].y);
}
Error:
cudaFree(dev_d);
cudaFree(dev_v);
cudaFree(dev_m);
cudaFree(dev_v0);
cudaFree(dev_d0);
return cudaStatus;
}</code></pre>Okay, there's quite a bit here to go through, but it's pretty straightforward. We're doing all of the GPU device setup here as well as iterating the simulation, so the device setup happens first. We declare the device variables, set which device we're using, allocate all of the memory on the device, and copy the data to the device for the input vectors. Again, I showed the first example of error handling and omitted the rest. Yes, this code is using gotos to jump to the end of the function on an error and free up any memory that has been allocated. That's how the template did it, and I just stuck with it. It could be easily changed by making one more level of function call that returned the error code and the memory could be freed at that level to keep everything clean and goto-free. I figured this use of gotos was pretty innocuous for a short piece of code.<div><br /></div><div>Now we're at the crux of the function—the iterations. The number of iterations is actually doubled in this for loop, and the reason to call <span style="font-family: courier;">gravityIter()</span> twice in this way is that it makes it easy to ping-pong between the current and next velocity and position vectors. Notice that <span style="font-family: courier;">dev_v</span> and <span style="font-family: courier;">dev_d</span> are swapped with <span style="font-family: courier;">dev_v0</span> and <span style="font-family: courier;">dev_d0</span> between the two calls. This swap makes the previous call's new velocity and position vectors the next call's initial velocity and position vectors, and the other vectors can be used for the new new data that will be calculated. At the top of the next iteration, the variables get swapped again automatically. No temporary variables necessary!</div><div><br /></div><div>The last thing I want to mention is the <span style="font-family: courier;">printf()</span> statements. They're used to get the positions out to the debug console for, well, debugging purposes, and I can graph the data in a spreadsheet for an easy visual display. However, this method of getting data out of the program will not be sustainable because printing to the console in the middle of a simulation is <i>very </i>slow. We'll cook up something better in the future.</div><div><br /></div><h4 style="text-align: left;">Initializing the Simulation</h4><div>All that's left to do is initialize some variables in <span style="font-family: courier;">main()</span> and call that last <span style="font-family: courier;">gravitySim()</span> function. I saved the easiest for last:</div>
<pre><code class="cpp">int main()
{
const int arraySize = 2;
const float m[arraySize] = { 1, 2 };
const vec2d d0[arraySize] = { {0, 0}, {1, 0} };
const vec2d v0[arraySize] = { {0, 0}, {0, 1} };
vec2d d[arraySize] = { {0, 0}, {0, 0} };
const float dt = 1;
const unsigned int iterations = 10;
// Run gravity simulation.
cudaError_t cudaStatus = gravitySim(d, m, v0, d0, dt, arraySize, iterations);
if (cudaStatus != cudaSuccess) {
fprintf(stderr, "gravitySim failed!");
return 1;
}
// cudaDeviceReset must be called before exiting in order for profiling and
// tracing tools such as Nsight and Visual Profiler to show complete traces.
cudaStatus = cudaDeviceReset();
if (cudaStatus != cudaSuccess) {
fprintf(stderr, "cudaDeviceReset failed!");
return 1;
}
return 0;
}
</code></pre>These variable initializations are meant to create a simple simulation that I can easily see is correct. The masses of the bodies are so small that they should have no effect on the acceleration or velocity of either body, and I should see the first body stay at (0, 0) while the second body flies upward (in the y-direction) at 1 m/s. Since dt is 1 second, it should reach y = 20 m. And we can see it works perfectly:<div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFvxPn86Swb61Ps0WGWzlTiHuvZMbxSTqTrBwEkKsjJOtTM_ZeUe7F2_ve9ZUXEtkf9iRR4XVsf-s2M94-FwKEQQWx0NCpli3uIXMP8Kltq0YrgLCnrQhZpKbYhGvRL85gVRdG6WOb4II/s659/vs_debug_console_gravity_sim_test.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Console snapshot of successful gravity sim test" border="0" data-original-height="290" data-original-width="659" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFvxPn86Swb61Ps0WGWzlTiHuvZMbxSTqTrBwEkKsjJOtTM_ZeUe7F2_ve9ZUXEtkf9iRR4XVsf-s2M94-FwKEQQWx0NCpli3uIXMP8Kltq0YrgLCnrQhZpKbYhGvRL85gVRdG6WOb4II/s16000/vs_debug_console_gravity_sim_test.png" /></a></div><br /><div>Cool beans, it worked! I bet you were surprised. Time to try something more ambitious.</div><div><br /></div><h4 style="text-align: left;">Earth Orbiting the Sun</h4><div>Let's try plugging in the values for the mass of the Sun and the Earth, the average distance of the Earth from the Sun in the x direction, and the average velocity of the Earth in the y direction (values found at Wikipedia):</div>
<pre><code class="cpp"> const float m[arraySize] = { 1.989e30, 5.972e24 };
const vec2d d0[arraySize] = { {0, 0}, {1.496e11, 0} };
const vec2d v0[arraySize] = { {0, 0}, {0, 2.978e4} };
vec2d d[arraySize] = { {0, 0}, {0, 0} };
const float dt = 86400;
const unsigned int iterations = 183;
</code></pre>Mass is in kg, distance in meters, velocity in m/s, and time in seconds. Those values are a heckuva lot bigger than before, and they should produce a gravitational effect that will be seen in the calculations. These values give a step time of one day, and a simulation time of just over one year, because the iterations are doubled. So what do we get?<div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXL2ZO6AlHvfCtjA_DSJmHBWkQjKIUA_vguyi6ftWThkAdKJh51PHAjr-xtZO2EXGdT6j140z8TIrkDV6A-t6RqH6lmKJGQeVK6ZGvcpbkUMNtFuG97QaMaYO9788gLwHqd9Y2UXp_Y4Y/" style="margin-left: 1em; margin-right: 1em;"><img alt="Plot of Earth's orbit with 1 day time step" data-original-height="623" data-original-width="600" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXL2ZO6AlHvfCtjA_DSJmHBWkQjKIUA_vguyi6ftWThkAdKJh51PHAjr-xtZO2EXGdT6j140z8TIrkDV6A-t6RqH6lmKJGQeVK6ZGvcpbkUMNtFuG97QaMaYO9788gLwHqd9Y2UXp_Y4Y/s16000/Earth%2527s+Orbit+%2528dt+%253D+1+day%2529.png" /></a></div><br />Well, um. That doesn't look quite right. I mean, the Earth is definitely going around the Sun, and the Sun is staying put in the center (at least at this scale). Yet, the Earth doesn't complete a full orbit in a year, and it kind of looks like it's going to spiral out from there. What's the problem? Well, it turns out that simulations that use differential equations are extremely dependent on the resolution of the time step, among other things. This happens to be a simulation of differential equations. You can tell by the <span style="font-family: courier;">dt</span> variable. That delta-time is a dead giveaway that we're dealing with a difficult simulation problem.</div><div><br /></div><div>We can increase the accuracy of the simulation by decreasing the time step. That means, instead of assuming that the Earth is going to maintain its calculated acceleration and velocity for a full day, we're going to narrow that timeframe to six hours—a quarter of a day. In reality, the Sun's gravity has a continuous effect on the Earth, and vice versa, so the acceleration and velocity are changing continuously, but I don't have all day to run this simulation. I just want to see if we can get an improvement. Here's the exact same simulation with a time step of 21,600 seconds and 732 (really 1464) iterations:</div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipiHRLjocLyxa015KzVc-zNhqb0VrYfFxSYBHY58RqFhUzac4e1iLE-_Scy0lDT8VhMIJzNEkR9P1rp8xweV1GAj_3KHdZhGCKoAYxWwA-WP3TxoQaFYMViE1vwGBS2B_jn-OSTjD6oLQ/" style="margin-left: 1em; margin-right: 1em;"><img alt="Earth's orbit with 6 hour time step" data-original-height="606" data-original-width="600" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipiHRLjocLyxa015KzVc-zNhqb0VrYfFxSYBHY58RqFhUzac4e1iLE-_Scy0lDT8VhMIJzNEkR9P1rp8xweV1GAj_3KHdZhGCKoAYxWwA-WP3TxoQaFYMViE1vwGBS2B_jn-OSTjD6oLQ/s16000/Earth%2527s+Orbit+%2528dt+%253D+0.25+day%2529.png" /></a></div><br />This is looking promising. We still don't complete a full orbit, but we're much closer. I'm sure dropping the time step to half an hour or ten minutes or something will get us close enough for practical purposes. What about that Sun-dot in the middle; what does that look like?</div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxFj7nqw_9LwecKfBLBMYdr2xRXOOuzuJDfu8fx085wqSrVAmEN-dwulDBqHtYELsU8vEUkJtFtyNsSUMznkbr6uV3HIlGVU32ivunbKWts1YyEW9YNtZE00KfhVvByGo1BZ8Lg6cKLGc/" style="margin-left: 1em; margin-right: 1em;"><img data-original-height="371" data-original-width="600" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxFj7nqw_9LwecKfBLBMYdr2xRXOOuzuJDfu8fx085wqSrVAmEN-dwulDBqHtYELsU8vEUkJtFtyNsSUMznkbr6uV3HIlGVU32ivunbKWts1YyEW9YNtZE00KfhVvByGo1BZ8Lg6cKLGc/s16000/Sun%2527s+Motion.png" /></a></div><br /><div>What the heck is that?! It looks like the Sun swings out 800,000 meters (800 km) and comes back 300 km "up" from where it started. I know the Earth exerts some pull on the Sun, along with all of the other planets, that causes it to wobble around, but I would expect it to look more like a circle. The issue here is likely initial conditions. The Sun shouldn't start motionless, and in fact, it should start with an initial velocity in the negative y direction, opposite the Earth's velocity. That should give this motion a more circular path. Another possible way to see that circular path is to extend the simulation for much longer. The Sun and the Earth should settle into a stable orbit with the Sun having a small circular wobble. This simulation shows another important point about these types of simulations, namely that the initial conditions matter–a lot. I know I've seen these things mentioned in books and on websites, but it really brings it home to see it in a simulation you've built yourself. Another thing that I expect has an effect, although smaller than the time step or initial conditions for this particular simulation, is the finite resolution of the 32-bit floats. The resolution of the data, and the rounding error of the calculations could be improved by using doubles, but the trade-off is a slower simulation. That resolution problem is going to have an effect at extremely long time scales, which is why it's so hard to simulate the solar system accurately out to more than a few thousand years. For our purposes, I'll stick with floats until it becomes obvious that we need the extra resolution.</div><div><br /></div><div><br /></div><div>This was all very exciting to get a simulation up and running on my graphics card using CUDA. I would like to explore this simulation of Earth's orbit some more, but it's tedious to keep exporting the data from the console to a spreadsheet and plotting it. Next time we'll look at plotting the simulation in realtime (or at least simtime) with a visual display. I mean, I'm already programming the graphics card, might as well use it for graphics, too. The visual display will also enable us to explore more complicated simulations in the future. Right now the simulation isn't quite right for more than two bodies, but that will come after we can see it on the screen.</div></div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-6601237195174373042020-12-07T20:40:00.001-06:002020-12-07T22:55:11.463-06:00Playing With Gravity to Learn CUDA: A Fundamental Force<p>We started out this <a href="https://sam-koblenski.blogspot.com/2020/11/playing-with-gravity-to-learn-cuda.html">series on learning CUDA</a> by diving in and writing a couple CUDA programs that we then got up and running on an nVidia graphics card. That was a great start that gave us an immediate feeling of accomplishment, but to keep advancing toward our goal of building a multi-body gravity simulation, we're going to have to take a break from CUDA and make sure we understand gravity a bit more. Gravity can be modeled at different levels of complexity, so we'll want to decide at what level we want to model it. We'll certainly start simple, but it's still good to know enough about gravity to know where we could go if we wanted and where it's not worth it to explore.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEib2SdLkBjyK9fXllTagqZ3CHkAgqE-2RG3qQcrG4inUMwUqm6v_2F0jtkZ4cDGOmhRtg6-nz4wOR6ia6KdgsB3poTnWezviRUA98FRCFPcUDt5KH10hFNL-Q66qiil_Ese9NL71NCvfws/" style="margin-left: 1em; margin-right: 1em;"><img alt="Orbiting neutron stars with gravitational waves" data-original-height="1080" data-original-width="1921" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEib2SdLkBjyK9fXllTagqZ3CHkAgqE-2RG3qQcrG4inUMwUqm6v_2F0jtkZ4cDGOmhRtg6-nz4wOR6ia6KdgsB3poTnWezviRUA98FRCFPcUDt5KH10hFNL-Q66qiil_Ese9NL71NCvfws/w400-h225/gravitational_waves.jpeg" width="400" /></a></div><br /><p></p><span><a name='more'></a></span><h4 style="text-align: left;">What is Gravity?</h4><div>While scientists don't know exactly what gravity is—whether it's associated with an undiscovered particle, governed by a field, or purely a property of spacetime—we do know it is one of four fundamental forces in the universe. Gravity along with the forces of electromagnetism and the weak and strong nuclear forces govern all known interactions that we can observe between matter and radiation, and gravity happens to be the weakest of these forces. This fact may seem surprising, but consider that even a relatively weak fridge magnet can overcome the force of gravity generated by the entire planet Earth, and you get a sense of how weak gravity really is.</div><div><br /></div><div>While gravity may be weak, it's effects are the most far-reaching of the four forces. The weak and strong nuclear forces are limited to the scale of the nucleus of an atom, so their effects are extremely local. Electromagnetism is a polarized force, having positive and negative charges. Over large distances these opposite charges tend to balance, cancelling out the effect of the force at those distances. Arguably, photons carry the electromagnetic force as radiation over large distances, but the effect of the force is still localized relative to the photon. Gravity, on the other hand, doesn't have a polarity and it only decays with the square of the distance, so the effects of a gravitational field can reach much farther through space than the other forces.</div><div><br /></div><div>In more technical terms, gravity is an attractive force between objects that varies proportionately with the mass of those objects and how far apart they are from each other. More massive objects spaced more closely together will generate a stronger force between those objects. This conception of gravity is the Newtonian theory of gravity. Einstein added to Newton's theory with the theory of general relativity, which more accurately describes the properties and behaviors of gravity. Instead of the concept of masses acting directly on each other at a distance through an instantaneous force, general relativity introduces the idea that masses cause spacetime to curve, and curved spacetime determines how objects move through it. </div><div><br /></div><div>General relativity has additional implications for objects in motion. Instead of objects in orbit, like planets, revolving around their parent stars for all eternity, planets will actually dissipate energy (extremely slowly) into the fabric of spacetime, and their orbits will decay over very, very long timescales. Gravity will also affect massless objects, like photons, because the curvature of spacetime will affect the path of a photon and cause it to curve around large masses when it gets near them due to the large curvature of spacetime around those large masses. General relativity still makes all of the same calculations and predictions that Newtonian gravity does, but it extends into realms where Newtonian gravity either can't cope or gets things wrong.</div><div><br /></div><h4 style="text-align: left;">How Should We Model Gravity?</h4><div>We have a lot of potential things to model here with our gravity simulation. Objects could have multiple kinds of forces affecting them, and gravity can have secondary effects due to Einstein's general relativity. However, we're going to simplify things both by the nature of what we're going to model and because simpler calculations will allow us to scale to bigger systems. </div><div><br /></div><div>Because gravity is such a weak force that doesn't have noticeable effects until the masses are planetary-sized or larger, we'll be modeling systems with objects and distances on at least solar system scales. At those scales the electromagnetic, weak nuclear, and strong nuclear forces are negligible, so we're going to ignore them. We'll also start with the simplifying assumption that objects will be point-masses with no size to themselves. We may try to model masses with size eventually, but it turns out that at stellar and definitely interstellar distances, most objects are effectively point-masses anyway.</div><div><br /></div><div>We're also going to ignore general relativity and instead stick with Newtonian gravity for this model. While general relativity is much more accurate when it comes to describing the cosmos, it really comes into play with extremely massive and dense objects, like black holes and neutron stars, or over extremely long timescales, like the <a href="https://en.wikipedia.org/wiki/Tests_of_general_relativity#Perihelion_precession_of_Mercury" target="_blank">perihelion precession of Mercury</a>. Because we're already modeling objects as point-masses and not over long enough timescales, we'll start with Newtonian gravity.</div><div><br /></div><div>The other major thing we have to decide how to deal with is time. For a multi-body system, each object's mass is going to affect every other object continuously over time, but we're simulating with the finite resources of a computer so we won't be able to do continuous calculations exactly. We'll simplify by calculating the forces on each object all at once in the same instant in time, and then calculate where those objects will be in space one timestep in the future. The size of the timestep can be configurable, and it will be a trade-off between accuracy and total simulated time.</div><div><br /></div><div>Oh, one last thing. We'll start with a 2D simulation. We may add a third dimension later because expanding the calculations to 3D is straightforward, but the additional calculations are time-consuming. We should get a good sense of how the systems work without the third dimension, and we'll be able to scale up to larger systems by working in only two dimensions.</div><div><br /></div><h4 style="text-align: left;">The Mathematics of Gravity</h4><div>We're going with the Newtonian definition of gravity in 2D, so what is that going to look like for the actual simulation? We need to express the definition of gravity in terms that a computer can calculate. That means a mathematical equation. As it turns out, we have one of those:</div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg27dPY-DFMzJvZhuioiyExwCzdboTKeBYJhUkM9Rw2P_8tTfayyzCqHi2Sm29QvNlkbaNP_tXr7xDsG5tgN0jvBJMGcwbOiF_zajTE_abAwS1vRcALXN40KFXtRy-6hVhBOmvSRBPvlQ0/" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1344" data-original-width="1920" height="224" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg27dPY-DFMzJvZhuioiyExwCzdboTKeBYJhUkM9Rw2P_8tTfayyzCqHi2Sm29QvNlkbaNP_tXr7xDsG5tgN0jvBJMGcwbOiF_zajTE_abAwS1vRcALXN40KFXtRy-6hVhBOmvSRBPvlQ0/" width="320" /></a></div><br /><i>F</i> is the force on the corresponding mass <i>m</i>. <i>G</i> is the gravitational constant, <span face="sans-serif" style="background-color: white; color: #202122; font-size: 14px; white-space: nowrap;">6.674</span><span face="sans-serif" style="background-color: white; color: #202122; font-size: 14px; margin-left: 0.25em; margin-right: 0.15em; white-space: nowrap;">×</span><span face="sans-serif" style="background-color: white; color: #202122; font-size: 14px; white-space: nowrap;">10</span><sup style="background-color: white; color: #202122; font-family: sans-serif; font-size: 11.2px; line-height: 1; white-space: nowrap;">−11</sup><span face="sans-serif" style="background-color: white; color: #202122; font-size: 14px; white-space: nowrap;"> m</span><sup style="background-color: white; color: #202122; font-family: sans-serif; font-size: 11.2px; line-height: 1; white-space: nowrap;">3</sup><span face="sans-serif" style="background-color: white; color: #202122; font-size: 14px; white-space: nowrap;">⋅kg</span><sup style="background-color: white; color: #202122; font-family: sans-serif; font-size: 11.2px; line-height: 1; white-space: nowrap;">−1</sup><span face="sans-serif" style="background-color: white; color: #202122; font-size: 14px; white-space: nowrap;">⋅s</span><sup style="background-color: white; color: #202122; font-family: sans-serif; font-size: 11.2px; line-height: 1; white-space: nowrap;">−2</sup>. And <i>r</i> is the distance between the two masses. We'll need to calculate this as a vector for two dimensional space, so the equation has an additional vector term in practice:</div><div><br /></div><div style="text-align: center;"><img alt="{\displaystyle \mathbf {F} _{21}=-G{m_{1}m_{2} \over {\vert \mathbf {r} _{21}\vert }^{2}}\,\mathbf {\hat {r}} _{21}}" aria-hidden="true" class="mwe-math-fallback-image-inline" src="https://wikimedia.org/api/rest_v1/media/math/render/svg/7e50f0023475d4142a475fbf3ce58e839ad2e032" style="background-color: white; border: 0px; color: #202122; display: inline-block; font-family: sans-serif; font-size: 14px; height: 5.843ex; vertical-align: -3.005ex; width: 20.918ex;" /></div><div><br /></div><div>Where r̂<sub>21</sub> is the unit vector from object 1 to object 2. There would also be an equal but opposite force <b>F</b><sub>12</sub> from object 2 to object 1. Once we have all of these forces between objects calculated and summed up for each object, we need to translate the forces into motion of the objects. This is Newton's second law of motion:</div><div><br /></div><div style="text-align: center;"><b>F</b> = <i>m</i><b>a</b></div><div style="text-align: center;"><b><br /></b></div><div style="text-align: left;">Where <b>a</b> is another vector for acceleration in the same direction as <b>F</b>. We can use the calculated acceleration vectors and the known velocity and position vectors from the the previous timestep to arrive at the velocity and position vectors for the timestep being calculated:</div><div style="text-align: left;"><br /></div><div style="text-align: center;"><b>v</b> = <b>v</b><sub>0</sub> + <b>a</b><i>t</i></div><div style="text-align: center;"><i><br /></i></div><div style="text-align: center;"><b>d</b> = <b>d</b><sub>0</sub> + <b>v</b><i>t</i> + 0.5<b>a</b><i>t</i><sup>2</sup></div><div style="text-align: center;"><br /></div><div style="text-align: left;">These equations give us the position of every object in the simulation at each timestep, which is all we need to run the simulation. We'll need to provide some initial conditions for <b>v</b><sub>0</sub> and <b>d</b><sub>0</sub> for every object at the start of the simulation to kick-start the first timestep, but after that, the simulation will determine these initial conditions for every subsequent timestep.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><div style="text-align: left;">Now that we understand a bit more about gravity and how we'll simplify things to build a tractable simulation, we're ready to see if we can actually do it. Next time we'll put these equations to use and see if we can write a program that simulates at least two massive objects using CUDA on the GPU.</div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-25890250513701059902020-11-15T21:41:00.003-06:002020-11-16T18:34:55.742-06:00Playing With Gravity to Learn CUDA: Hello, World<p>I have been interested in CUDA for a long time, and while I've read a few books on CUDA, I have not actually gotten my hands dirty with it, yet. It's time to remedy that situation with this new blog series where I explore CUDA programming with a purpose. The end goal is to create a CUDA program that runs a scalable, multi-body gravity simulation on my GPU. Maybe I'll even draw what's being simulated on the screen. I'm not sure at this point how big I'll be able to make this simulation, but I think it'll be a good way to dive into CUDA and see what it's all about. The real goal here is to experiment, fail, struggle, and learn while having some fun doing it.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnfHMSB4DnLxpdw8UaIBYqfw6MfaVhpB4WaipKq1zNKUpduR1W48m9_7Qn7uzz6UFezMJIaSGhEo9qkSUAhMCbOnXZe4MELPkmBC_mM_0GiPNlXffK77ZVnAV3Fp7ofwlhYL5c5lhXwVk/" style="margin-left: 1em; margin-right: 1em;"><img alt="Representation of Sun-Earth Gravity" data-original-height="389" data-original-width="744" height="209" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnfHMSB4DnLxpdw8UaIBYqfw6MfaVhpB4WaipKq1zNKUpduR1W48m9_7Qn7uzz6UFezMJIaSGhEo9qkSUAhMCbOnXZe4MELPkmBC_mM_0GiPNlXffK77ZVnAV3Fp7ofwlhYL5c5lhXwVk/w400-h209/gravity_resize_md.jpg" width="400" /></a></div><br /><p></p><span><a name='more'></a></span><h4 style="text-align: left;"><span>In the Beginning There Was Hello, World</span></h4><script>hljs.initHighlightingOnLoad();</script><div><span>First things first, we need to figure out how to run a CUDA program on our GPU, and since the best program to start with is Hello, World, that is where we'll begin. I have an nVidia GeForce GTX 1050 card, so the following experimentation in drivers and the environment will all correspond to that GPU. Different cards may need different driver versions or other setup minutiae to get them working, so YMMV. Having said that, I found the CUDA environment setup to be pretty painless, at least on Windows. To set up your own CUDA environment, just follow these steps:</span></div><div><ol style="text-align: left;"><li><span>Download and install <a href="https://visualstudio.microsoft.com/" target="_blank">Visual Studio Community 2019</a> (or later versions). Yes, the CUDA environment integrates with VS, so you get all of the code editing and debugging tools from Microsoft. If you have the Pro or Enterprise versions, good for you; those work, too.</span></li><li><span>Download and install <a href="https://developer.nvidia.com/cuda-downloads" target="_blank">CUDA Toolkit 11.1</a> (or later versions).</span></li><li><span>While steps 1 and 2 take a really long time, there is no step 3.</span></li></ol><div>I can't say much about the process on Linux, since I don't have a Linux machine running with a CUDA-capable card. Actually, I can say that I have a friend who did some work on a video image recognition ML project using CUDA on Linux, and he said the setup was a nightmare of aligning Linux distro version, toolkit version, and driver version with his card. I can imagine that getting messy, but you also might get lucky with a well-supported setup. It looks like nVidia does have a number of officially supported setups for Linux on their <a href="https://developer.nvidia.com/cuda-downloads" target="_blank">CUDA Toolkit download page</a>. Anyway, those are the joys of programming environment setup for you.</div></div><div><br /></div><div>Getting back to the task at hand, we just need to fire up VS and create a new project using the CUDA 11.1 Runtime template:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLHXF1StDCuEO22jxSZnWu1tDC8IuaHPGn92qAinEVKgppOtwPlP0HtZRl8XSAv2ZApctTn79WCM8lyhO4KWX4dYjBh_9Y3-obtYUpKoS71UbZQNt_AWeYTFbhdyTYlUYNiYzIeOcYtn0/s1024/visual_studio_new_project_cuda_template.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Visual Studio Create a new project with CUDA 11.1 Runtime template screenshot" border="0" data-original-height="680" data-original-width="1024" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLHXF1StDCuEO22jxSZnWu1tDC8IuaHPGn92qAinEVKgppOtwPlP0HtZRl8XSAv2ZApctTn79WCM8lyhO4KWX4dYjBh_9Y3-obtYUpKoS71UbZQNt_AWeYTFbhdyTYlUYNiYzIeOcYtn0/w400-h265/visual_studio_new_project_cuda_template.png" width="400" /></a></div><br /><div>Click "Next" and enter a project name like "hello_world." Change the location if you wish, and then click "Create." Wait a few minutes for VS to do its thing (at least on my decrepit machine; yours is probably faster), and we're presented with a working project that adds two arrays of numbers using CUDA. While this type of program is probably a more appropriate first CUDA program, we're going to ignore it for now, delete everything except the initial #includes, and replace it with this code:</div>
<pre><code class="cpp">#include "cuda_runtime.h"
#include "device_launch_parameters.h"
#include <stdio.h>
__global__ void helloFromGPU(void) {
printf("Hello, World from GPU!\n");
}
int main() {
printf("Hello, World from CPU!\n");
helloFromGPU<<<1, 10>>>();
cudaDeviceReset();
}</code></pre>
<div>As we can plainly see from <span style="font-family: courier;">main()</span>, this program should at least print out "Hello, World from CPU!" But then it does this weird function call with a <span style="font-family: courier;"><<<1, 10>>></span> stuck in between the function name and the parentheses. What that call does is load that function, referred to as a kernel in CUDA-speak, on the GPU and replicate it on 10 identical threads that run on 10 identical cores (we'll get to what the "1" means eventually, but not now). That means we should see 10 printouts of "Hello, World from GPU!" If we click the play button for the Local Windows Debugger, we'll see exactly that:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdkkn_OoWNZuWnqFHfF9hnhFGT_jQZaDKMKDD2YGw1hBHVdAVeE7SC37lRfqy3SaLatv7mMd74l76JaiijG3Dkfjdo2BPT-Q7U142-y2CJxiqNrA5e8md76LT4C3DN8SAPrFYIZZsMh50/s475/vs_debug_console_hello_world.png" style="margin-left: 1em; margin-right: 1em;"><img alt="VS Debug Console for Hello, World screenshot" border="0" data-original-height="173" data-original-width="475" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdkkn_OoWNZuWnqFHfF9hnhFGT_jQZaDKMKDD2YGw1hBHVdAVeE7SC37lRfqy3SaLatv7mMd74l76JaiijG3Dkfjdo2BPT-Q7U142-y2CJxiqNrA5e8md76LT4C3DN8SAPrFYIZZsMh50/s16000/vs_debug_console_hello_world.png" /></a></div><br /><div>That is pretty slick! We have our first CUDA program running on the GPU using multiple cores, and it even does printing to the console. Alright, now that we know our environment is set up properly, let's go back to the template program that VS had created for us and we had summarily deleted and see what that's all about.</div><div><br /></div><h4 style="text-align: left;">A Real First CUDA Program</h4><div>Hello, World doesn't really do CUDA programming justice—it's just printing instead of computation—so we're going to figure out what's going on behind the scenes with a more interesting program. Here's the template program that VS creates for us with a new project:</div>
<pre><code class="cpp">#include "cuda_runtime.h"
#include "device_launch_parameters.h"
#include <stdio.h>
cudaError_t addWithCuda(int *c, const int *a, const int *b, unsigned int size);
__global__ void addKernel(int *c, const int *a, const int *b) {
int i = threadIdx.x;
c[i] = a[i] + b[i];
}
int main() {
const int arraySize = 5;
const int a[arraySize] = { 1, 2, 3, 4, 5 };
const int b[arraySize] = { 10, 20, 30, 40, 50 };
int c[arraySize] = { 0 };
// Add vectors in parallel.
cudaError_t cudaStatus = addWithCuda(c, a, b, arraySize);
printf("{1,2,3,4,5} + {10,20,30,40,50} = {%d,%d,%d,%d,%d}\n",
c[0], c[1], c[2], c[3], c[4]);
cudaStatus = cudaDeviceReset();
return 0;
}
// Helper function for using CUDA to add vectors in parallel.
cudaError_t addWithCuda(int *c, const int *a, const int *b, unsigned int size) {
int *dev_a = 0;
int *dev_b = 0;
int *dev_c = 0;
cudaError_t cudaStatus;
// Choose which GPU to run on, change this on a multi-GPU system.
cudaStatus = cudaSetDevice(0);
// Allocate GPU buffers for three vectors (two input, one output) .
cudaStatus = cudaMalloc((void**)&dev_c, size * sizeof(int));
cudaStatus = cudaMalloc((void**)&dev_a, size * sizeof(int));
cudaStatus = cudaMalloc((void**)&dev_b, size * sizeof(int));
// Copy input vectors from host memory to GPU buffers.
cudaStatus = cudaMemcpy(dev_a, a, size * sizeof(int), cudaMemcpyHostToDevice);
cudaStatus = cudaMemcpy(dev_b, b, size * sizeof(int), cudaMemcpyHostToDevice);
// Launch a kernel on the GPU with one thread for each element.
addKernel<<<1, size>>>(dev_c, dev_a, dev_b);
// cudaDeviceSynchronize waits for the kernel to finish, and returns
// any errors encountered during the launch.
cudaStatus = cudaDeviceSynchronize();
// Copy output vector from GPU buffer to host memory.
cudaStatus = cudaMemcpy(c, dev_c, size * sizeof(int), cudaMemcpyDeviceToHost);
cudaFree(dev_c);
cudaFree(dev_a);
cudaFree(dev_b);
return cudaStatus;
}</code></pre>
<div>This program takes two vectors of five elements each and adds them together. The trick is that each addition of a pair of elements happens on a different core in the GPU because they are each done in their own thread. The setup of these threads is all done in the function <span style="font-family: courier;">addWithCuda()</span>, and the kernel is called about midway down this function with <span style="font-family: courier;">addKernel<<<1, size>>>()</span>. The <span style="font-family: courier;">size</span> parameter is the size of the arrays, so a unique thread is created for each element in the arrays. The other statements are allocating memory on the GPU, moving vectors back and forth between the main memory and GPU, and freeing the memory on the GPU. I left the comments in so the steps are fairly self-explanatory in the code.</div><div><br /></div><div>I did, however, strip out the error reporting to make the steps a bit more clear. Everywhere that a CUDA function returns a value that's assigned to <span style="font-family: courier;">cudaStatus</span>, this return value could potentially be an error that should be checked and reported for easier debugging.</div><div><br /></div><div>Now, looking at the <span style="font-family: courier;">addKernel</span> function, we can see a new variable called <span style="font-family: courier;">threadIdx.x</span>. This variable isn't declared anywhere in this program because it's part of the CUDA runtime. It gives each thread running on the GPU access to the index it's doing its calculations for, and from the <span style="font-family: courier;">.x</span> element access we can gather that there are potentially other dimensions to the thread index. This calculation only uses one-dimensional arrays, but CUDA can handle 2D data as well, where the second dimension index is accessed with the obvious <span style="font-family: courier;">threadIdx.y</span>. This feature makes 2D calculations much more convenient.</div><div><br /></div><div>Okay, if we click the play button again, we can see this program run as well:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZ9W3MeKLkkC5wpyLDI0yHTD3026xsVWrjdt8XLcoDyVNY72Ha1PC-TnuG2__Rlxt5gCOfOWef4uM3135VEWvneVeWav9olFrHGvlapR9QiCpo0annNQ3yTNHczJlakc0XjjrAhWDmi7U/s659/vs_debug_console_addWithCuda.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of addWithCuda template program" border="0" data-original-height="54" data-original-width="659" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZ9W3MeKLkkC5wpyLDI0yHTD3026xsVWrjdt8XLcoDyVNY72Ha1PC-TnuG2__Rlxt5gCOfOWef4uM3135VEWvneVeWav9olFrHGvlapR9QiCpo0annNQ3yTNHczJlakc0XjjrAhWDmi7U/s16000/vs_debug_console_addWithCuda.png" /></a></div><br /><div>Alright, very exciting; we can add! So what else can we do with this CUDA environment?</div><div><br /></div><h4 style="text-align: left;">What's In This GPU Anyway?</h4><div>To take full advantage of whatever GPU hardware we're running on, and write scalable programs that can optimally run on different levels of hardware, we need to know what our GPU hardware is capable of. Fortunately, nVidia has already thought of this, and even more fortunately, they have provided a set of example programs and optimized libraries with the CUDA toolkit that includes a program that queries the CUDA devices present in the system. If we load the VS solution that by default is installed at C:\ProgramData\NVIDIA Corporation\CUDA Samples\v11.1\Samples_vs2019.sln, we can find the <span style="font-family: courier;">deviceQuery</span> project under the <span style="font-family: courier;">1_Utilities</span> folder in the Solution Explorer.</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeWh0GhMbJnMWdRc5SVgkQMTT0FIUzC3fghyphenhyphenCHqC6VGUt0K-Jr8m6r3czoBRN-knjpRQaJOEYNyXSQMnwRHxEHcP89gHs-LTm9g5xNqdgxC0nn1-cIN8lL534u5r1Evbfof8aSFUGmaF4/s479/vs_solution_explorer_deviceQuery.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of deviceQuery in VS Solution Explorer" border="0" data-original-height="479" data-original-width="358" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeWh0GhMbJnMWdRc5SVgkQMTT0FIUzC3fghyphenhyphenCHqC6VGUt0K-Jr8m6r3czoBRN-knjpRQaJOEYNyXSQMnwRHxEHcP89gHs-LTm9g5xNqdgxC0nn1-cIN8lL534u5r1Evbfof8aSFUGmaF4/s16000/vs_solution_explorer_deviceQuery.png" /></a></div><br /><div>This program is much lengthier than the others, and more boring. It's mostly a bunch of queries of the CUDA devices in the system and printouts of what it finds. It's hardly worth showing. There's not even any kernel, as all of the info can be gleaned from the host thread, but here's a snippet from <span style="font-family: courier;">main()</span> to get a sense of how the code works:</div>
<pre><code class="cpp">int main(int argc, char **argv) {
pArgc = &argc;
pArgv = argv;
printf("%s Starting...\n\n", argv[0]);
printf(
" CUDA Device Query (Runtime API) version (CUDART static linking)\n\n");
int deviceCount = 0;
cudaError_t error_id = cudaGetDeviceCount(&deviceCount);
if (error_id != cudaSuccess) {
printf("cudaGetDeviceCount returned %d\n-> %s\n",
static_cast<int>(error_id), cudaGetErrorString(error_id));
printf("Result = FAIL\n");
exit(EXIT_FAILURE);
}
// This function call returns 0 if there are no CUDA capable devices.
if (deviceCount == 0) {
printf("There are no available device(s) that support CUDA\n");
} else {
printf("Detected %d CUDA Capable device(s)\n", deviceCount);
}
int dev, driverVersion = 0, runtimeVersion = 0;
for (dev = 0; dev < deviceCount; ++dev) {
cudaSetDevice(dev);
cudaDeviceProp deviceProp;
cudaGetDeviceProperties(&deviceProp, dev);
printf("\nDevice %d: \"%s\"\n", dev, deviceProp.name);
// Console log
cudaDriverGetVersion(&driverVersion);
cudaRuntimeGetVersion(&runtimeVersion);
printf(" CUDA Driver Version / Runtime Version %d.%d / %d.%d\n",
driverVersion / 1000, (driverVersion % 100) / 10,
runtimeVersion / 1000, (runtimeVersion % 100) / 10);
printf(" CUDA Capability Major/Minor version number: %d.%d\n",
deviceProp.major, deviceProp.minor);
char msg[256];
sprintf_s(msg, sizeof(msg),
" Total amount of global memory: %.0f MBytes "
"(%llu bytes)\n",
static_cast<float>(deviceProp.totalGlobalMem / 1048576.0f),
(unsigned long long)deviceProp.totalGlobalMem);
printf("%s", msg);</code></pre>
<div>And it goes on and on. Notice that we're able to grab a lot of device properties from the library call <span style="font-family: courier;">cudaGetDeviceProperties()</span>, and those properties can be used to decide how we want to partition our problem when running on different GPUs. Some of these queries will be very useful for optimizing algorithms in a GPU-agnostic way. We can run this program on our system right now to see what its actual capabilities are as well, but this program should be built as a release build (not debug!) and run from the command prompt for it to work. When it's done, I get this for my GTX 1050:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZ91tFgquBChZ2XIzq8WFv-TGJLBrFS-LVKjBzcCf7GmePtvKJUBgKshjlb8BlBQx8BkUsTMsRQ6AiDSobl6k3vQkuKgTcl0JcAMYymREy2WvgMxwd3V4N7sQRF3-vFNrurZIIvLiekGA/s905/vs_console_deviceQuery.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of CUDA deviceQuery program" border="0" data-original-height="638" data-original-width="905" height="283" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZ91tFgquBChZ2XIzq8WFv-TGJLBrFS-LVKjBzcCf7GmePtvKJUBgKshjlb8BlBQx8BkUsTMsRQ6AiDSobl6k3vQkuKgTcl0JcAMYymREy2WvgMxwd3V4N7sQRF3-vFNrurZIIvLiekGA/w400-h283/vs_console_deviceQuery.png" width="400" /></a></div><br /><div>I can see that it is indeed a GeForce GTX 1050 card with 2GB of memory running at 3.5GHz, 640 CUDA cores running at 1.5GHz, and a CUDA capability of 6.1, along with many other parameters. That CUDA capability is important for knowing what CUDA features are available on the hardware. Each GPU generation adds more features, and knowing the CUDA capability can allow programs to use those features conditionally for better performance or fall back on a different solution if the desired feature isn't available. Okay, this is all great stuff, but let's ends with something a bit more flashy.</div><div><br /></div><h4 style="text-align: left;">Wow Me</h4><div>There are dozens of sample programs provided in this toolkit, so I wanted to take a look at something a bit more impressive. Looking in the 2_Graphics folder, the Mandelbrot project caught my eye. This code is significantly more involved than the other code we've looked at, so I'm not going to analyze it here. Just know that the Mandelbrot set is an infinite fractal that can be drawn and zoomed in on to see all sorts of interesting patterns. The calculations for coloring the pixels are also very parallelizable. This program once again built and ran without a hitch, and here is what it looks like:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_uGiPZilWWJdYrBntdHMkLfacrlQDGuqfZrAOboyxvN_tk67bB7IniMiALrrn2ws6Toye4R7twjv5NTs1qj7DkaQnd2qyWVuZvMvVzrFmWwe0PqmKeGgemHuhfw5wVrAeFoFenoyjG7Q/s786/cuda_mandlebrot_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of Mandelbrot set at full view" border="0" data-original-height="616" data-original-width="786" height="314" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_uGiPZilWWJdYrBntdHMkLfacrlQDGuqfZrAOboyxvN_tk67bB7IniMiALrrn2ws6Toye4R7twjv5NTs1qj7DkaQnd2qyWVuZvMvVzrFmWwe0PqmKeGgemHuhfw5wVrAeFoFenoyjG7Q/w400-h314/cuda_mandlebrot_screenshot.png" width="400" /></a></div><br /><div>And here's another view after zooming in on one of those filament-like structures off of the circle to the left of the main blob:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigSIhXQL269MKwRESk3AgtE-CAh_fCSNSXP94DmQeuBxvBAIyn6X7teePL6J22YHO-SvcMG-FxN3xohDoDE-IB-w92UbSgrnT0GJe8M3XeYSSnr9vNwuMEXVtMum1qCNI2pXmx2UC7-pU/s786/cuda_mandlebrot_screenshot_zoom.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of Mandelbrot set zoomed in" border="0" data-original-height="616" data-original-width="786" height="314" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigSIhXQL269MKwRESk3AgtE-CAh_fCSNSXP94DmQeuBxvBAIyn6X7teePL6J22YHO-SvcMG-FxN3xohDoDE-IB-w92UbSgrnT0GJe8M3XeYSSnr9vNwuMEXVtMum1qCNI2pXmx2UC7-pU/w400-h314/cuda_mandlebrot_screenshot_zoom.png" width="400" /></a></div><br /><div>Huh, looks like there's another similar shape to the full Mandelbrot set hiding inside that filament. There's actually more if we zoom into the filaments seen here or even go further into the Mandelbrot shape in the center and find filaments in there to zoom in on. There's all kinds of patterns that repeat at essentially infinite zoom levels. This particular Mandelbrot viewing program is a bit rough, so it can't zoom to the extreme magnitudes that a more optimized program could. But we can see some extremely cool examples of it on YouTube with better color rendering:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="266" src="https://www.youtube.com/embed/8cgp2WNNKmQ" width="320" youtube-src-id="8cgp2WNNKmQ"></iframe></div><br /><div>These Mandelbrot zooms can get real trippy real fast, and there's a ton of them, so be careful going down this YouTube rabbit hole. Even though the sample program doesn't look as good as this, it's still awesome to see something this complex running on my own GPU with less than an hour of effort.</div><div><br /></div><div><br /></div><div>This is a pretty good place to stop for this episode. We got a Hello, World program running with CUDA, saw how a real CUDA program works to add two vectors in parallel, learned how to query the GPU for all of its specs, and even ran a fairly complex and computation-intensive parallel program on the GPU. That's a solid start on my goal of writing a multi-body gravity simulation, and there's probably some other useful stuff in the sample programs provided with the CUDA toolkit. I'll definitely be able to use some of the drawing code in the samples to make a visual representation of the sim. That will have to wait for another time, though. Next episode we'll be learning about gravity.</div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-60478161333584279492020-10-26T20:16:00.001-05:002020-10-26T20:17:35.433-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2: Dogfooding<p>This is it―the last post of this series on exploring the monster taming mechanics of Final Fantasy XIII-2 is finally here, and in it we're actually going to do what the title says. Now that we've <a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">integrated all of the tables of data</a> that we extracted from the FFXIII-2 FAQs into a functional website, we're ready to use that website to explore how to optimize the monster taming as we play through the game. This exercise is known in the tech industry as dogfooding, where we proceed to eat our own dog food, so to speak, to see whether or not the product we're building is any good. During the course of using the website, we may find opportunities for improvement, missing features, and other things that could just be done better. It's an excellent way to iterate on a product to improve it.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkuMWugYKRgmoyP54n7PpxuMS3oG0LSYPbFQpuGCbdm5g3bMcIWsHTqSwj5OoLqMZtslcPSjoKd9FHAgo6CIAV-UTvHyWn1txn6m1k_8C4fmQa_2-dl4QNW0m2TarcuGc5_C7Yfvn9Ssc/s1280/ffxiii-2_odin.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Final Fantasy XIII-2 monster taming Odin" border="0" data-original-height="720" data-original-width="1280" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkuMWugYKRgmoyP54n7PpxuMS3oG0LSYPbFQpuGCbdm5g3bMcIWsHTqSwj5OoLqMZtslcPSjoKd9FHAgo6CIAV-UTvHyWn1txn6m1k_8C4fmQa_2-dl4QNW0m2TarcuGc5_C7Yfvn9Ssc/w400-h225/ffxiii-2_odin.jpg" width="400" /></a></div><span><a name='more'></a></span><h4 style="text-align: left;">The Early Game</h4><div>We don't get to start monster taming right from the outset in FFXIII-2. There's a fair amount of intro game to play through before we get to the monster taming partway through Bresha Ruins 005 AF. Once we gain that ability, we'll want to start looking for the monsters that will most help us get through the game, and more importantly, focus on upgrading those monsters instead of wasting monster materials on monsters that we're not going to keep for very long. On the other hand, it's okay to upgrade some of the early monsters that may not max out at a high level if they're especially strong to start with. These early strong monsters can be used as a bridge to the even stronger mid and late game monsters that will need to be leveled up before they're ready to fight.</div><div><br /></div><div>The easiest way to get a sense of which monsters we want to go after is to go to the Role Abilities page and select an ability that's common to every monster in one of the six classes. For example, selecting the Attack ability gives a list of Commando monsters sorted by their location depth:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYNbayWPPfEiXUFIfDhEZC4OU0w-J9eRxg0Y8xHukR-v0fRoKA66gqdlHepgGqOK9mvFmZ0JIK-mmS89wH88QUuutwcttS4rEafBjx9HXEu6X036-h0y4HEReO7jVavKTcG0kbwKZz8KA/s1290/ffxiii-2_commando_monsters_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of Commando monsters" border="0" data-original-height="652" data-original-width="1290" height="203" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYNbayWPPfEiXUFIfDhEZC4OU0w-J9eRxg0Y8xHukR-v0fRoKA66gqdlHepgGqOK9mvFmZ0JIK-mmS89wH88QUuutwcttS4rEafBjx9HXEu6X036-h0y4HEReO7jVavKTcG0kbwKZz8KA/w400-h203/ffxiii-2_commando_monsters_screenshot.png" width="400" /></a></div><br /><div>We can see a few options here. Right in the level where we can start monster taming, we can capture Meonekton, Hoplite, and Uridimmu (also Svarog, but not until we complete the story for the level). Of those monsters, Hoplite is the clear choice because of its high base strength and "very mercurial" growth rate. Even though it can only get to level 20, it's okay to dump materials into upgrading a few early monsters because we'll collect plenty of those low-level materials throughout the normal course of the game. Behemoth in the next area of Yaschas Massif 010 AF is a nice longer-term monster, if you manage to beat enough of them to tame one or get lucky. They're a tough beast this early in the game, and in the level at this stage you're actually trying to avoid battles with them. They're also a very late bloomer, so we won't be able to upgrade it beyond the strength of Hoplite until later in the game, when other stronger monsters from the Behemoth family are available. Bottom line, Hoplite is monster we want for our Commando right now.</div><div><br /></div><div><b>Ravager: </b>For early spell-casters we have the following:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMl9YmCjXmlt1PllWn9Ewcf3BBef3QGmLzyw2ncnW0g2i1JdM6mEOjhjI1bugwMSrxDsu0FOlfbZAFpiqPtYVjY5yHBAf81clzyQNL_s2kOR356kLBWUa-UnYpsylGb0QCQUi9uQHyj_0/s1290/screenshot_ravager_monsters.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot Ravager monsters" border="0" data-original-height="595" data-original-width="1290" height="185" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMl9YmCjXmlt1PllWn9Ewcf3BBef3QGmLzyw2ncnW0g2i1JdM6mEOjhjI1bugwMSrxDsu0FOlfbZAFpiqPtYVjY5yHBAf81clzyQNL_s2kOR356kLBWUa-UnYpsylGb0QCQUi9uQHyj_0/w400-h185/screenshot_ravager_monsters.png" width="400" /></a></div><br /><div>As you can see, there are a couple Commando monsters mixed in here because there's a slight deficiency in selecting Ravagers―there's no common role ability that would select them all like there is with Attack for Commandos. This could be fixed by adding a filtered link to the monster table to filter by role. Here, I just selected the Bresha Ruins 005 AF location. We don't see more interesting Ravagers for quite a while anyway so looking only in Bresha Ruins 005 AF is sufficient, but it would be convenient to add that role filtering ability as an exercise.</div><div><br /></div><div>We start out with a Zwerg Scandroid that's kind of a wimp, but we should hold off on upgrading it or any other Ravagers that we may catch until we can snag an Albino Lobo after completing the story events of Bresha Ruins 005 AF. This monster has the ability to learn all of the main elemental spells to the second level and their weapon buff equivalents (e.g. Froststrike), and it starts with decent magic and growth rate. It'll be a pretty good Ravager for the first half of the game.</div><div><br /></div><div><b>Sentinel: </b>The Sentinels can be filtered for with the provoke or steelguard role abilities:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_VHVHWWD8HiZHcpD5UK60Xdi9xnvO2r6IxY_6syXiyupDpE5fuiX9m_WSixnyLlmyTTX5FFc3WTCPJCH1YJkzA9_TFpAeCxWXzvVsONsmmFtvRuK3amOjNt8ug_mctY1OHoPFwtpiQbA/s1290/screenshot_sentinel_monsters.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot Sentinel monsters" border="0" data-original-height="437" data-original-width="1290" height="135" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_VHVHWWD8HiZHcpD5UK60Xdi9xnvO2r6IxY_6syXiyupDpE5fuiX9m_WSixnyLlmyTTX5FFc3WTCPJCH1YJkzA9_TFpAeCxWXzvVsONsmmFtvRuK3amOjNt8ug_mctY1OHoPFwtpiQbA/w400-h135/screenshot_sentinel_monsters.png" width="400" /></a></div><br /><div>The obvious choice is the Pulse Knight since it has high base strength, it learns HP +10% at level 3, and it's the only option in Bresha Ruins 005 AF. The Pulse Knight will be a stalwart companion for a fair amount of time if for no other reason than the fact that there are so few Sentinel monsters to choose from.</div><div><br /></div><div><b>Saboteur: </b>The Saboteur can be found from the deprotect or deshell role abilities, at least, any monster we <i>care</i> about has those abilities:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFMtZay4lviM9mKWUGCV2Uzs5BGRXMV4cYibLStT66uviFxiy8PODftBKNMIjZXW74RKOb55MfwmWw97kJ_l-8D9g7cSiLYh9amIISVe7JnXyfHU0ZPLptSXI0df183odYGKQm_je7Zw0/s1279/screenshot_saboteur_monsters.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot Saboteur monsters" border="0" data-original-height="661" data-original-width="1279" height="206" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFMtZay4lviM9mKWUGCV2Uzs5BGRXMV4cYibLStT66uviFxiy8PODftBKNMIjZXW74RKOb55MfwmWw97kJ_l-8D9g7cSiLYh9amIISVe7JnXyfHU0ZPLptSXI0df183odYGKQm_je7Zw0/w400-h206/screenshot_saboteur_monsters.png" width="400" /></a></div><br /><div>This is it. Only seven Saboteur monsters have the deprotect ability. It looks like we're going with the Unsaganashi since we can't get Rangda until we backtrack to New Bodhum 003 AF with the moogle throw ability towards mid-game. There's no need to upgrade Unsaganashi, though, since we'll be able to snag Chelicerata before too long, and that's the Saboteur we ultimately want.</div><div><br /></div><div><b>Synergist:</b> The Synergists better have bravery or faith buffs, so let's filter on bravery:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4yhVxSJWvYNrvZh2z1kt7aNT4iZy-t8EybaU9lSyl9Fn_xVE5nXssIR0Kj-94Rkye8k1EAyXibGMeh1vPwOf-piVS0Px0oRcF_wTNmejhyphenhyphenMfwZaINnfeTBdm-dFFFFivrokqvBRlkJ1o/s1288/screenshot_synergist_monsters.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot Synergist monsters" border="0" data-original-height="658" data-original-width="1288" height="204" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4yhVxSJWvYNrvZh2z1kt7aNT4iZy-t8EybaU9lSyl9Fn_xVE5nXssIR0Kj-94Rkye8k1EAyXibGMeh1vPwOf-piVS0Px0oRcF_wTNmejhyphenhyphenMfwZaINnfeTBdm-dFFFFivrokqvBRlkJ1o/w400-h204/screenshot_synergist_monsters.png" width="400" /></a></div><br /><div>Like the Saboteurs, there are not many Synergists to choose from. At first there's only Gahongas, so it's lucky that it has decent base magic and speed, and it starts with bravery and learns faith at level 6. Gahongas will work out nicely until we can get a Purple Chocobo in Bresha Ruins 300 AF, not too much further into the game. The Purple Chocobo is nice because it has a bit more HP and strength so it can hold its own in battle for a while, and it isn't as slow to upgrade as the Gahongas, so save most of the monster materials for the chocobo.</div><div><br /></div><div><b>Medic:</b> A good Medic is literally a life saver, and it's going to have cure, so that's our filter:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlcxC1a5Cx-6gZp-lhLloshFk7XGGewWZg_Q9XqOUcoZUD2uMgYpA2U0UYrC-7FjgdGfE-gd2ykF2OWZOWZ36uK_AY6gSfNn3hhhrRF651lZGXPkDutCBr2XbHBGmG-0q6AzvmzJpihgc/s1285/screenshot_medic_monsters.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot Medic monsters" border="0" data-original-height="665" data-original-width="1285" height="208" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlcxC1a5Cx-6gZp-lhLloshFk7XGGewWZg_Q9XqOUcoZUD2uMgYpA2U0UYrC-7FjgdGfE-gd2ykF2OWZOWZ36uK_AY6gSfNn3hhhrRF651lZGXPkDutCBr2XbHBGmG-0q6AzvmzJpihgc/w400-h208/screenshot_medic_monsters.png" width="400" /></a></div><br /><div>Cait Sith will likely be captured in short order after we first learn about monster taming, and it'll be the best medic we have for some time. It's probably worth upgrading at least until it learns Esuna so that it can cure ailments, and it might be worth going all the way to level 21 for Curasa if you have enough materials on hand.</div><div><br /></div><h4 style="text-align: left;">The Middle Game</h4><div>Around the time we get to the Archylte Steppe can be considered the mid game, and we start to have a lot more options for monster taming. We'll be able to nab some monsters that will be useful through the rest of the game, and we'll have to make some decisions on what's best to upgrade with our limited mid-level materials, assuming you're like me and hate grinding to acquire all of the things. We'll browse through each of the roles again, but without posting screenshots of every list. Most of the screenshots above show middle game monsters anyway.</div><div><br /></div><div><b>Commando:</b> We definitely have more choices for Commandos at this point in the game. One of the coolest monsters to pick up is the albino behemoth Narasimha in Yaschas Massif 010/01X AF and about equally good is the Reaver in the Archylte Steppe. Here's an instance where it's hard to make a choice based on the information we have in the tables. Narasimha has a base strength of 179 and a mercurial growth rate, but Reaver has a base strength of 132 and a very mercurial growth rate. They both can level up to 70, so which one is better? We don't know. At least, we don't know from the monster table. We'd need to upgrade each monster in the game to see which one was better and when Reaver would pass up Narasimha, if it does. That would be nice additional info to have, but it's not readily available online and I'm not going to itemize it all for this post. In the end, both of them are great, strong beasts, and you can pick your favorite.</div><div><br /></div><div><b>Ravager:</b> A great Ravager for the mid game is surely the Blue Chocobo. Chocobos are just awesome, and this one is nice and quick. It's a late bloomer, so it'll require a bit of upgrading before it reaches its true potential, but it should start getting there with the materials available mid game.</div><div><br /></div><div><b>Sentinel:</b> This next Sentinel will be the last one you ever need. The Pulse Gladiator is a wall when you get it, and it becomes a fortress after upgrading. There's not much more to say here; easy choice.</div><div><br /></div><div><b>Saboteur: </b>A surprisingly good Saboteur is available in both the Archylte Steppe and Oerba 200 AF, which can be reached at the same point in the game. Chelicerata looks like a wimp with a base magic of 47, but it has a very mercurial growth rate and can go all the way up to level 99. It comes with deshell II, learns deprotect at level 4, and learns all the multi-target debuffs by the end. It may not look great when you first get it, but throw some materials at it and see how it can shine in the middle game.</div><div><br /></div><div><b>Synergist: </b>There aren't any better options than the Purple Chocobo we already have, so just keep upgrading it instead of looking elsewhere.</div><div><br /></div><div><b>Medic: </b>Yay, another chocobo! We'll run into the Green Chocobo in Yaschas Massif 110 AF, and be able to replace the Cait Sith kitty with a much stronger giant chicken. It's fast, like all of the chocobos, it learns raise at level 22, which is very helpful, and it can level up to 99, so we can keep it through the rest of the game if we want.</div><div><br /></div><h4 style="text-align: left;">The Late Game</h4><div>Towards the end of the game, we'll find a few more interesting monsters and be able to upgrade some of our favorites to their highest levels, unlocking their true potential. Here's a quick rundown of the best monsters in the game from searching through the monster table and detail views.</div><div><br /></div><div><b>Commando:</b> Surprisingly, the best late game Commando is the small, unassuming Chichu. Square has a habit of doing things like this, making super powerful characters or items out of things that look dinky. Why is Chichu so good? I mean, it starts out with only 156 strength, which, granted, is better than Reaver's 132, but for a late game monster, we would expect it to start a bit higher. Well, it's a late bloomer so we don't see it's true potential until we've fully upgraded it to level 70. By then it's also wicked fast and has great abilities like armor breaker, bravery feeder, faith feeder, vigilance feeder, and strength +25% so it can debuff it's opponents while buffing itself and beating down everything in sight. It's a…well…it's a monster.</div><div><br /></div><div><b>Ravager:</b> It's best to stick with the Blue Chocobo for a late game Ravager, since it can be upgraded to level 99 and gets a ton of great abilities along the way, along with 6 ATB slots that allow it more attacks per ATB charge. All chocobos except for the Golden and Silver Chocobos can be upgraded to level 99, so that's why they tend to be great monsters to use.</div><div><br /></div><div><b>Sentinel:</b> Yeah, we'll stick with the Pulse Gladiator and just level it up, thank you very much.</div><div><br /></div><div><b>Saboteur: </b>The Black Chocobo, which we don't find until the Vile Peaks 010 AF, is a great late game Saboteur with all of the debuffs you would want in one: deprotega, woundga, heavy imperilga, and heavy painga. Like most chocobos, it levels to 99.</div><div><br /></div><div><b>Synergist: </b>Keep upgrading that Purple Chocobo, and it'll keep you happy.</div><div><br /></div><div><b>Medic: </b>We're pretty chocobo-heavy at the end of the game, but the Green Chocobo is still the best option here, so get it up to level 99 to reach its full potential.</div><div><br /></div><div><br /></div><div>This exercise in dogfooding the monster taming website we built worked out pretty well. We found that overall the site is pretty usable, and we charted a nice path of monster taming throughout the game. We also found a few improvements along the way that would sweeten the experience even more. The monster table could use a filter on the role column, a few other columns could use a sort feature, and it would be great to have a few more columns for mid and late level stats. Although, adding those stats would be a fair amount of work. Another feature that would be nice to have is some tooltips on the abilities, which would also involve adding descriptions to the role abilities. Oh, and thumbnail pics of all the monsters would liven up the monster table and details pages. While all of these features would make the site even better, it's definitely usable as-is. Just browsing through it and clicking around on all of the monsters and locations makes me want to pick up the game again for another play through. Now I have all the information I need right at my fingertips to make it smooth sailing.</div><div><br /></div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-29795157434725330142020-10-03T21:14:00.003-05:002020-10-26T20:20:30.205-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2: Monster CharacteristicsWe've done something with just about all of the data we've extracted on monster taming in this series on exploring Final Fantasy XIII-2 monster taming mechanics—except for the monster characteristics. Last time we finally integrated <a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">the monster materials table into our website</a>, so now we're ready to tackle the last orphaned data. But what should we do with these monster characteristics? They each consist of only a name and a description, so it seems kind of boring to just link their names in the monster table to the monster characteristics table. We should be able to do something more useful with them. Conveniently, Bootstrap has a tooltip component that would be a perfect use for this data, so we're going to explore that idea.<div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYYjsTVb_2UNi68MLaLRZy-99TmLPPQ1qPGmj7AZlH-PvPPcVqWSD0kodJZUcCiraov5Ae84QhLPPRTiATXpYGQDDE-T8sAIsehYvFX-AVL_yj3jLi-M5OWDS-UYkBNDJ_XdL1vHJHTm4/s300/ffxiii-2_svarog.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Final Fantasy XIII-2 Svarog" border="0" data-original-height="168" data-original-width="300" height="224" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYYjsTVb_2UNi68MLaLRZy-99TmLPPQ1qPGmj7AZlH-PvPPcVqWSD0kodJZUcCiraov5Ae84QhLPPRTiATXpYGQDDE-T8sAIsehYvFX-AVL_yj3jLi-M5OWDS-UYkBNDJ_XdL1vHJHTm4/w400-h224/ffxiii-2_svarog.jpg" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div>
<a name='more'></a><h4>
Creating Tooltips</h4>
<script>hljs.initHighlightingOnLoad();</script>
Tooltips are a nice little user interface feature that pops up a descriptive box when the user hovers their mouse over a target element on the page. We have just such a set of elements that would warrant a description in the "Growth" column of the monster table. The values of "early peaker," "well-grown," and "late bloomer" are not self-explanatory, so we want to provide descriptions that will better inform the user, if they're confused, with a convenient tooltip.<div><br /></div><div>Tooltips are a bit more involved than the other Bootstrap features we've used, so they need a little JavaScript initialization. We can see from the <a href="https://getbootstrap.com/docs/4.4/components/tooltips/">Bootstrap documentation</a> that we have to include this little code snippet in our HTML to initialize tooltips on a page after the DOM loads:</div>
<pre><code class="js">$(function () {
$('[data-toggle="tooltip"]').tooltip()
})</code></pre>To get this to work in Rails, we can stick this JavaScript in a file that we can later reference in our views for the pages that we want tooltips to load. Let's put it in <span style="font-family: courier;">app/javascript/packs/monster_tooltips.js</span>. Next, we can test out the tooltips by adding the following code to our monster table view in <span style="font-family: courier;">app/views/monster/index.html.erb</span>:<br />
<pre><code class="html"><h1>Monsters</h1>
<%= javascript_pack_tag 'monster_tooltips' %>
<table id="monster-table" class="table table-striped table-sm">
<thead class="thead-dark">
<tr>
<th scope="col">Monster Name</th>
<th scope="col">Role</th>
<th scope="col">Location</th>
<th scope="col">Tame Rate</th>
<th scope="col">Max Level</th>
<th scope="col">Base HP</th>
<th scope="col">Base Strength</th>
<th scope="col">Base Magic</th>
<th scope="col">Speed</th>
<th scope="col"><span data-toggle="tooltip" data-original-title="Test Tooltip">Growth</span></th>
<th scope="col">Immune</th>
<th scope="col">Resistant</th>
<th scope="col">Halved</th>
<th scope="col">Weak</th>
</tr>
</thead></code></pre>This code snippet is just showing up to the header row of the table, and we can see the <span style="font-family: courier;">javascript_pack_tag</span> that's added to load the <span style="font-family: courier;">monster_tooltips.js</span> file. Then, in the header cells, the "Growth" column header is wrapped in a span that adds a <span style="font-family: courier;">data-toggle</span> property of "tooltip" and a <span style="font-family: courier;">data-original-title</span> property with the text we want to display in the tooltip. Now if we reload the monster table page and hover the cursor over the "Growth" header, we see our tooltip pop up:<div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjs3K__xi5j1L_lDr-zYqPuBdGt3vwXLqmwbyBovoeLn-oQefAjh8axcHSd0Gt2xp0-RjyNGHS3IaD7buk6CcGucta_Ew72vSTGbs3ZhOso4kgiiNz89mSobz-AePAJ7ZRup_XA4bsR5zI/s1290/screenshot_monster_tooltip.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of tooltip on Growth header in monster table" border="0" data-original-height="471" data-original-width="1290" height="146" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjs3K__xi5j1L_lDr-zYqPuBdGt3vwXLqmwbyBovoeLn-oQefAjh8axcHSd0Gt2xp0-RjyNGHS3IaD7buk6CcGucta_Ew72vSTGbs3ZhOso4kgiiNz89mSobz-AePAJ7ZRup_XA4bsR5zI/w400-h146/screenshot_monster_tooltip.png" width="400" /></a></div><br /><div>Neato. Okay, now that we have that sorted out, all we have to do is wrap each growth cell in a span tag and put the correct text in, right?</div><div><br /></div><h4 style="text-align: left;">Adding Growth Tooltips</h4><div>It's not quite that simple because there are a couple problems. First, we need to connect the growth values with references to the monster characteristics table so we can get at the descriptions. That's not too hard. We've done <a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">that kind of thing before</a>. We need to change the data type of the <span style="font-family: courier;">growth</span> attribute to references in the monster table migration and set the foreign key to the characteristics table:</div>
<pre><code class="ruby">class CreateMonsters < ActiveRecord::Migration[6.0]
def change
create_table :monsters do |t|
t.string :name
t.string :role
t.references :location, foreign_key: true
t.references :location2, foreign_key: {to_table: :locations}
t.references :location3, foreign_key: {to_table: :locations}
t.integer :max_level
t.integer :speed
t.string :tame_rate
t.references :growth, foreign_key: {to_table: :characteristics}
t.string :immune
t.string :resistant
t.string :halved
t.string :weak
t.references :normal_drop, foreign_key: {to_table: :materials}
t.references :rare_drop, foreign_key: {to_table: :materials}
# ... many more attributes ...
end
end
end</code></pre>Then, we add the link to the monster model so we can get to the characteristic model from the growth attribute in the monster model. This code is added to <span style="font-family: courier;">app/models/monster.rb</span>:<br />
<pre><code class="ruby">class Monster < ApplicationRecord
belongs_to :location, class_name: 'Location', optional: true
belongs_to :location2, class_name: 'Location', optional: true
belongs_to :location3, class_name: 'Location', optional: true
belongs_to :growth, class_name: 'Characteristic'
belongs_to :normal_drop, class_name: 'Material', optional: true
belongs_to :rare_drop, class_name: 'Material', optional: true
# ... many more belongs_to macros ...
end</code></pre>Next, we add another one-liner near the end of <span style="font-family: courier;">db/seeds.rb</span> before <span style="font-family: courier;">Monster.create!(monster)</span> so we can set the growth references correctly in the monster table during the database import:<br />
<pre><code class="ruby"> monster['growth'] = Characteristic.find_by(name: monster['growth'])</code></pre>We're almost ready to rebuild the database, except for one catch. Not all of the growth values in the monster table exist in the characteristic table! If we do the import now, a bunch of the growth values are going to be nil. Don't worry, though. It's not too many values that we need to add to the <span style="font-family: courier;">db/monster_characteristics.csv</span> file to make everything right, so we'll just do it manually:<br />
<pre><code class="text">name,description
Early Peaker,Reaches a maximum level of 20.
Well-Grown,Reaches a maximum level of 30-60.
Standard,Reaches a maximum level of 30-60.
Very Mercurial,Reaches a maximum level of 20-99. Best stat gains happen very early.
Mercurial,Reaches a maximum level of 70-99. Best stat gains happen early.
Late Bloomer,Reaches a maximum level of 70-99.
Very Late Bloomer,Reaches a maximum level of 70-99. Best stat gains happen late.
Erratic,Reaches a maximum level of 70-99. Best stat gains are random.
Very Erratic,Reaches a maximum level of 70-99. Stat gains are very erratic.</code></pre>I guess the FAQ writers felt the need to add a few more growth categories, and this is my best guess at what they meant. Anyway, now we should be able to rebuild the database:<br />
<pre><code class="csh">$ rails db:drop
$ rails db:migrate
$ rails db:seed</code></pre><div style="text-align: left;">After correcting one misspelling of "Peaker" as "Peeker" for PuPu, we have the updated database with references from the monster table to the characteristic table.</div><div style="text-align: left;"><br /></div><h4 style="text-align: left;">Viewing Monster Tooltips</h4><div>We're finally ready to add the growth tooltips to the monster table page. We already added the required <span style="font-family: courier;">javascript_pack_tag</span>, but the code is fairly short so I'll just show the whole thing:</div>
<pre><code class="html"><h1>Monsters</h1>
<%= javascript_pack_tag 'monster_tooltips' %>
<table id="monster-table" class="table table-striped table-sm">
<thead class="thead-dark">
<tr>
<th scope="col">Monster Name</th>
<th scope="col">Role</th>
<th scope="col">Location</th>
<th scope="col">Tame Rate</th>
<th scope="col">Max Level</th>
<th scope="col">Base HP</th>
<th scope="col">Base Strength</th>
<th scope="col">Base Magic</th>
<th scope="col">Speed</th>
<th scope="col">Growth</th>
<th scope="col">Immune</th>
<th scope="col">Resistant</th>
<th scope="col">Halved</th>
<th scope="col">Weak</th>
</tr>
</thead>
<% @monsters.each do |monster| %>
<tr>
<td><%= link_to monster.name, monster_path(monster) %></td>
<td><%= monster.role %></td>
<td>
<%= link_to monster.location.name, location_index_path(:highlight => monster.location.name) %>
<% if monster.location2 %>
<%= link_to monster.location2.name, location_index_path(:highlight => monster.location2.name) %>
<% end %>
<% if monster.location3 %>
<%= link_to monster.location3.name, location_index_path(:highlight => monster.location3.name) %>
<% end %>
</td>
<td><%= monster.tame_rate %></td>
<td><%= monster.max_level %></td>
<td><%= monster.maximum_base_hp %></td>
<td><%= monster.maximum_base_strength %></td>
<td><%= monster.maximum_base_magic %></td>
<td><%= monster.speed %></td>
<td>
<span data-toggle="tooltip" data-original-title="<%= monster.growth.description %>">
<%= monster.growth.name %>
</span>
</td>
<td><%= monster.immune %></td>
<td><%= monster.resistant %></td>
<td><%= monster.halved %></td>
<td><%= monster.weak %></td>
</tr>
<% end %>
</table></code></pre>All we had to do was wrap a span tag around the growth name and add the growth description as the tooltip. We also needed to make sure to change <span style="font-family: courier;">monster.growth</span> to <span style="font-family: courier;">monster.growth.name</span> because <span style="font-family: courier;">monster.growth</span> is now a reference to the element in the characteristic model. I should also note that in general, tooltips do not need to be attached to span tags. Almost any type of tag will do, although more complex tags that have other bootstrap features attached to them may misbehave. When that happens, wrapping a div or span tag around the element and attaching the tooltip to that tag instead will normally clear things up.<div><br /></div><div>So now we have tooltips on growth categories:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVM2tc8K3XItE-59VipYvxAgItFJHFSztLBEJpZrmZOL0QUi_zSdYwdAB5YQeWRLqSq7IOCFZiPcHqYHVJPnifhRnfqSD2VOjQLUbb5lnK087PKD7DYExgiHBCl6ZD26ypyaL3hOggPLo/s1289/screenshot_monster_tooltip_view.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of monster table with tooltip on growth category" border="0" data-original-height="467" data-original-width="1289" height="145" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVM2tc8K3XItE-59VipYvxAgItFJHFSztLBEJpZrmZOL0QUi_zSdYwdAB5YQeWRLqSq7IOCFZiPcHqYHVJPnifhRnfqSD2VOjQLUbb5lnK087PKD7DYExgiHBCl6ZD26ypyaL3hOggPLo/w400-h145/screenshot_monster_tooltip_view.png" width="400" /></a></div><br /><div>That's pretty slick. Now it's easy to add the same tooltips to the monster details page, and we want to go in there anyway to fix the <span style="font-family: courier;">monster.growth</span> reference so let's change <span style="font-family: courier;">app/views/monster/show.html.erb</span>:</div>
<pre><code class="html"><h1><%= @monster.name %></h1>
<%= javascript_pack_tag 'monster_tooltips' %>
<table id="monster-table" class="table table-striped table-sm">
<tr>
<td><strong>Role</strong></td>
<td><%= @monster.role %></td>
<td/>
<td/>
</tr>
<tr>
<td><strong>Monster Type</strong></td>
<td><%= @monster.monster_type %></td>
<td><strong>Constellation</strong></td>
<td><%= @monster.constellation %></td>
</tr>
<tr>
<td><strong>Location</strong></td>
<td>
<%= link_to @monster.location.name, location_index_path(:highlight => @monster.location.name) %>
<% if @monster.location2 %>
<br /><%= link_to @monster.location2.name, location_index_path(:highlight => @monster.location2.name) %>
<% end %>
<% if @monster.location3 %>
<br /><%= link_to @monster.location3.name, location_index_path(:highlight => @monster.location3.name) %>
<% end %>
</td>
<td><strong>Tame Rate</strong></td>
<td><%= @monster.tame_rate %></td>
</tr>
<tr>
<td><strong>Max Level</strong></td>
<td><%= @monster.max_level %></td>
<td><strong>Growth</strong></td>
<td>
<span data-toggle="tooltip" data-original-title="<%= monster.growth.description %>">
<%= @monster.growth.name %>
</span>
</td>
</tr>
<!-- many more table rows -->
</table></code></pre>The changes are exactly the same as for the monster index view, and now we have tooltips on the details view as well:<div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoCZQBZdAb1e421pLogbI48SAiXVv_kL3CmiQjUFJngxzSi1VCllTy6QcdUVsEJSGEmAiTYTL6RMnytY50Dm_dXue-E1mXNQRju_aic3kK5hJZoDE-naqBWrGibo4gw1o9vM0fEWYiupM/s1292/screenshot_monster_detail_tooltip.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of monster detail view with growth tooltip" border="0" data-original-height="634" data-original-width="1292" height="196" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoCZQBZdAb1e421pLogbI48SAiXVv_kL3CmiQjUFJngxzSi1VCllTy6QcdUVsEJSGEmAiTYTL6RMnytY50Dm_dXue-E1mXNQRju_aic3kK5hJZoDE-naqBWrGibo4gw1o9vM0fEWYiupM/w400-h196/screenshot_monster_detail_tooltip.png" width="400" /></a></div><br /><div>These growth tooltips are all well and good, but you may be asking yourself, "What about those other monster characteristics?" Well, as it turns out, the other characteristics don't appear anywhere else in our tables so there's not much we can do with them. We could add more attributes to the monster details page that call out these other characteristics, like flameproof or hearty, but that will be a lot of work adding them to every monster for not much gain. The rest of the characteristics table can be used mostly as a reference for looking up the descriptions for the in-game characteristics, should we need to. Most of the remaining ones are pretty self-explanatory anyway.</div><div><br /></div><div><br /></div><div>With that task done, we've connected the last table of data in our monster taming database, and we have a pretty fully-featured website for exploring this data. We're almost done with this series, so next time we'll wrap things up by actually using this data to answer the big question: how can I optimize the monster taming as I play through Final Fantasy XIII-2?</div></div><div><br /></div><div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div></div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-30537650965172490262020-09-13T22:06:00.005-05:002020-10-26T20:20:17.151-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2: Finding Monster Materials<p>We've started <a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html" target="_blank">asking deeper questions</a> of the monster taming data in this Exploring Monster Taming Mechanics series, but so far we've ignored the fact that these monsters can level up once we've tamed them. Their starting stats and abilities will not stay the same while the monsters develop, so we should take that into account when we're trying to find the strongest monsters. Monsters that can level up to higher levels are going to end up stronger than other monsters that may start out stronger but can't level up as much. And the way these monsters level up is by applying certain materials dropped by defeating other monsters in battle. We need to know when and where we can get those materials, so that is the subject of this post.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijz3Hy0xodY-FnI7Svc41t7Bs1-pDlwU7Xt4gQv7SbT9hN_g-VMCf8w-2WodcybZ2cHcJjzC0kc_Y3_GstI0LtIoU5Kfafwg7fJTey5xBmu3-lopB0HiWeVK1dXQDxZ1q8ffBJkQnLmJE/s800/FFXIII-2-Battle-Scene-5.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Final Fantasy XIII-2 Battle Scene" border="0" data-original-height="450" data-original-width="800" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijz3Hy0xodY-FnI7Svc41t7Bs1-pDlwU7Xt4gQv7SbT9hN_g-VMCf8w-2WodcybZ2cHcJjzC0kc_Y3_GstI0LtIoU5Kfafwg7fJTey5xBmu3-lopB0HiWeVK1dXQDxZ1q8ffBJkQnLmJE/w400-h225/FFXIII-2-Battle-Scene-5.jpg" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div>
<a name='more'></a><h4>
Where Are The Monster Materials?</h4>
<script>hljs.initHighlightingOnLoad();</script>
We have this monster materials table that lists all of the materials used to level up our monsters. The table has each material's name, its type, and its grade where higher grades of material are used on higher level monsters. This table does not, however, tell you where to find these materials. Since these materials are dropped by defeating monsters in battle, we could add a list of monsters that drop each material to this material table. Adding the data this way would require us to add some number of monsters to each material record, making sure that there's room in the table for the material dropped by the most monsters. The alternative is to add a material drop attribute to each monster. Since each monster drops at most one type of material, we only need to add one more attribute to the monster table to get this info, and then we can add references to and from the monster material table, just like we've done for other attributes. Conveniently, the monster table also contains the location of each monster, so it will be easy to see where we need to go for each material once this data is linked together. This approach seems to be a bit more sane, so let's go with it.<div><br /></div><div>The first thing we need to do is figure out which monsters drop which materials, and to do that we're going to have to go back to parsing an FAQ text file. The goal is to grab the drop data out of the FAQ, add it to the monster CSV file, and reload the monster data into the database. The FAQ that we'll use here is the same one we've been using for most of this series, the <a href="https://gamefaqs.gamespot.com/ps3/619315-final-fantasy-xiii-2/faqs/63731">Monster Infusion FAQ by BMSirius and sakurayule</a>. It happens to also have the material drop information we need in the (shock!) monster locations/drops section.</div><div><br /></div><div>Now that we've found the data, we just need to extract it, and to do that we can modify the monster FAQ parser state machine we built way back in <a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">the Data Collection episode</a>. Since the drop section in the FAQ appears before the tamable monster list section that we've already written states for, we'll use the strategy of inserting states for collecting the drop data before the states for collecting the monster attributes. Where are we going to store this drop data so we can merge it with the monster data? Well, the data array that already exists is a fine place because it's already maintained through the state machine, and it happens to be empty while we're parsing through the drop section so let's just use what we've got. We'll make a hash table the first item in the array with the monster names as keys to hold the dropped items.</div><div><br /></div><div>Before we start adding states, though, we'll need some constants for the section tags and the regexes that we'll use. These constants will be added to the beginning of <span style="font-family: courier;">monster_taming_parser.rb</span>. Let's go with tags for the start and end of the drop section and regexes for the monster name, normal drop, and rare drop:</div>
<pre><code class="ruby">DROP_SECTION_TAG = "MLDETAV"
END_DROP_SECTION_TAG = "|~|~~~~~~~~~"
NEW_DROP_MONSTER_REGEX = /^([\w\-]+(?:\s\S+)?)/
NEW_DROP_REGEX = /^Normal Drop: (?:\d\d\d% - )?(\S+(?:\s\S+)*)/
NEW_RARE_DROP_REGEX = /^Rare Drop--: (?:\d[\d\.]\d% - )?(\S+(?:\s\S+)*)/</code></pre>
<div>The tags are pretty self-explanatory since "MLDETAV" is literally the section tag in the FAQ for the desired section, and the <span style="font-family: courier;">END_DROP_SECTION_TAG</span> is the first unique character string that occurs after the end of that section. Using the end tag allows us to be a little looser with our regexes because we won't be accidentally matching them in an intermediate section after the drop section and before we find the tamable monster list section. </div><div><br /></div><div>The <span style="font-family: courier;">NEW_DROP_MONSTER_REGEX</span> will match on and capture any one or two words, including hyphenated words, that start at the beginning of a line. Most of the stuff in the drop section that we don't want is indented, so this regex will skip over those indented lines. It would however match on the same lines as <span style="font-family: courier;">NEW_DROP_REGEX</span> and <span style="font-family: courier;">NEW_RARE_DROP_REGEX</span>, so we'll need to be careful to run the drop regexes first. The drop regexes match when the line starts with "Normal Drop:" or "Rare Drop--:". It will then accept an optional three digit percentage or an "x.x%" before finally capturing any words that appear at the end of the line, which would be the name of the item. The percentage is how often the drop occurs, but we don't care about that for this purpose, so the regex doesn't capture it. </div><div><br /></div><div>With these regexes defined, we can add the states that we need to find and parse the drop section. The regexes are going to go between the <span style="font-family: courier;">section_tag_found</span> state and the <span style="font-family: courier;">start</span> state. Remember, the states are defined in reverse order so that state names that are used in other states are already defined.</div>
<pre><code class="ruby">section_tag_found = lambda do |line, data|
if line.include? SECTION_TAG
return find_separator, data
end
return section_tag_found, data
end
new_drop = lambda do |line, data|
if NEW_DROP_REGEX =~ line
drop_match = NEW_DROP_REGEX.match(line)
data.first[data.last] = [drop_match[1]]
elsif NEW_RARE_DROP_REGEX =~ line
drop_match = NEW_RARE_DROP_REGEX.match(line)
monster = data.pop
data.first[monster] << drop_match[1]
elsif NEW_DROP_MONSTER_REGEX =~ line
monster_match = NEW_DROP_MONSTER_REGEX.match(line)
return new_drop, data if monster_match[1] == 'Note'
return new_drop, data << monster_match[1]
elsif line.include? END_DROP_SECTION_TAG
return section_tag_found, data
end
return new_drop, data
end
find_drop_separator = lambda do |line, data|
if line.include? MONSTER_SEPARATOR
return new_drop, data << {}
end
return find_drop_separator, data
end
drop_section_tag_found = lambda do |line, data|
if line.include? DROP_SECTION_TAG
return find_drop_separator, data
end
return drop_section_tag_found, data
end
start = lambda do |line, data|
if line.include? DROP_SECTION_TAG
return drop_section_tag_found, data
end
return start, data
end</code></pre>
<div>Alright, let's walk through this backwards, starting at <span style="font-family: courier;">start</span>. Instead of looking for the tamable monster section tag, we'll look for the <span style="font-family: courier;">DROP_SECTION_TAG</span>. When we find it, it's actually the tag in the table of contents, so we need to look for it again. When we find it the second time, we know we're actually in the drop section, so we can look for the <span style="font-family: courier;">MONSTER_SEPARATOR</span>. This string happens to be the same separator for both sections, so we can reuse it to find the first monster, and then we're into the table of monster drops themselves. We push the hash table onto the data array first so it'll be available to use in the next state.</div><div><br /></div><div>That next state is <span style="font-family: courier;">new_drop</span>, which looks for all of the pieces of the record for a drop: the normal drop item, the rare drop item, and the monster's name – in that order so we don't match on the monster's name for everything. During execution the <span style="font-family: courier;">NEW_DROP_MONSTER_REGEX</span> is still going to match on the monster's name because the other two regexes will not match. We push this monster name onto the data array so we can use it as a new key in the hash table for when the normal and rare drops match. Also, notice that we're careful to <i>not</i> push the 'Note' string onto the data list because there are some 'Note:' lines in this table that will match the <span style="font-family: courier;">NEW_DROP_MONSTER_REGEX</span>, but we don't want them clogging up the data array.</div><div><br /></div><div>Then, when the NEW_DROP_REGEX matches, we get the name of the item and create a new hash entry using the last thing in the data array, which is the monster's name, as a key, and a new array with the first element as the dropped item. Luckily, monsters only have up to one normal drop and one rare drop, so we can make this a fixed array of two elements, a 2-tuple if you will. When the NEW_RARE_DROP_REGEX matches, we do a similar exercise, except we actually pop the monster's name off of the data array because we don't need it anymore, and we push the rare drop item name onto the array that already exists for that monster name key, making it a 2-tuple. Finally, when we reach the end of the section by finding the <span style="font-family: courier;">END_DROP_SECTION_TAG</span>, we can move on to searching for the tamable monster section, and we're done finding all of the monster drops. Whew.</div><div><br /></div><div>After this state machine runs, we have a data array that consists of a hash table of monster names as keys and 2-tuples of normal and rare drops as values, followed by a list of all of the monster records. We want to merge the hash table and subsequent list of records, and we can easily do this like so:</div>
<pre><code class="ruby">drops, *monsters = data
drops.default = ['None', 'None']
data = monsters.map do |monster|
name = monster['Name']
monster['Normal Drop'] = drops[name].first
monster['Rare Drop'] = drops[name].last
monster
end</code></pre>
<div>The first line neatly separates the hash table into <span style="font-family: courier;">drops</span> and the rest of the data array into <span style="font-family: courier;">monsters</span>. Then we set a default 2-tuple for the drops hash table so that any monsters that don't exist in the drop table will have 'None' assigned for its drops. Finally, we run through the list of monsters and pull out the normal and rare drops from the hash table to assign them to attributes for the correct monster. </div><div><br /></div><div>One thing I did when running this script was to remove the default hash value so that I could verify that all of the missing monsters were, in fact, really missing. I did find a number of legitimately missing monsters for which we don't care about their drops: Chichu, Golden Chocobo, Silver Chocobo, Rangda, Leyak, Nanochu, Cactuarama, Cactuarina, Valfodr, and Sazh. I also found ten monsters that had mysteriously changed names:</div><div><ul style="text-align: left;"><li>Feral Behemoth 🠊 Behemoth</li><li>Greater Behemoth 🠊 Grand Behemoth</li><li>Munchkin Maestro 🠊 Munchkin Boss</li><li>Zwerg Metrodroid 🠊 Zwerg Metro</li><li>Metal Gigantuar 🠊 Metalligantuar</li><li>Pulsework Gladiator 🠊 Pulse Gladiator</li><li>Pulsework Knight 🠊 Pulse Knight</li><li>Pulsework Soldier 🠊 Pulse Soldier</li><li>Flowering Cactuar 🠊 Cactrot</li><li>Lieutenant Amodar 🠊 Amodar</li></ul></div><div>It kind of looks like the captured monsters have a 14 character name limit, but it is funny that the wild Feral Behemoth becomes a plain Behemoth and the Flowering Cactuar becomes Cactrot. Domestication at its finest. I changed the names in the FAQ to be the captured names so that these monsters would match up properly between the two tables. Anyway, with the merge done, the only thing left is to add the verification patterns, like what was done in the <a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html" target="_blank">Data Validation and Database Import episode</a>, for "Normal Drop" and "Rare Drop" attributes to the <span style="font-family: courier;">VALID_MONSTER</span> hash:</div>
<pre><code class="ruby">VALID_MONSTER = {
"Name" => PROPER_NAME_REGEX,
"Role" => PROPER_NAME_REGEX,
# ... All the other attributes. They aren't all PROPER_NAME_REGEX. Really ...
"Normal Drop" => PROPER_NAME_REGEX,
"Rare Drop" => PROPER_NAME_REGEX,
}</code></pre>And we can run the script to regenerate <span style="font-family: courier;">monsters.csv</span> with the drops included.<div><br /></div><h4 style="text-align: left;">Adding the Drops to the Monster Table</h4><div>We could do this addition one of two ways: we could create a new migration to add the drop attributes to the database model and create a new seed script to populate them or we could edit the migration and seed scripts we have and delete and recreate the database. Let's burn it down and rebuild.</div><div><br /></div><div>First, like we did in the <a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html" target="_blank">Relational Data episode</a>, we need to add the new attributes to the monster schema in <span style="font-family: courier;">db/migrate/<long_datecode>_create_monsters.rb</span>:</div>
<pre><code class="ruby">class CreateMonsters < ActiveRecord::Migration[6.0]
def change
create_table :monsters do |t|
t.string :name
t.string :role
t.references :location, foreign_key: true
t.references :location2, foreign_key: {to_table: :locations}
t.references :location3, foreign_key: {to_table: :locations}
t.integer :max_level
t.integer :speed
t.string :tame_rate
t.string :growth
t.string :immune
t.string :resistant
t.string :halved
t.string :weak
t.references :normal_drop, foreign_key: {to_table: :materials}
t.references :rare_drop, foreign_key: {to_table: :materials}
# ... many more attributes ...
end
end
end</code></pre>Note that the <span style="font-family: courier;">normal_drop</span> and <span style="font-family: courier;">rare_drop</span> attributes are created as references to the materials table because we want to link them that way. Also note that I refrained from naming <span style="font-family: courier;">normal_drop</span> just <span style="font-family: courier;">drop</span>. It doesn't seem right to call an attribute in a database table <span style="font-family: courier;">drop</span>, does it? Next, we can add the connections for these attributes to the Monster model in <span style="font-family: courier;">app/models/monster.rb</span>:
<pre><code class="ruby">class Monster < ApplicationRecord
belongs_to :location, class_name: 'Location', optional: true
belongs_to :location2, class_name: 'Location', optional: true
belongs_to :location3, class_name: 'Location', optional: true
belongs_to :normal_drop, class_name: 'Material', optional: true
belongs_to :rare_drop, class_name: 'Material', optional: true
# ... many more belongs_to declarations ...
end</code></pre>These declarations are where we get the <span style="font-family: courier;">normal_drop</span> and <span style="font-family: courier;">rare_drop</span> method names that we can later use in the controller, view, and db seed script, so they really need to be there right away before we import the data into the database. I forgot this and was confused for a while when my seed script didn't work. We also add the connections in the opposite direction to the Material model in <span style="font-family: courier;">app/models/material.rb</span>:
<pre><code class="ruby">class Material < ApplicationRecord
has_many :normal_drop_monsters, :class_name => 'Monster', :foreign_key => 'normal_drop'
has_many :rare_drop_monsters, :class_name => 'Monster', :foreign_key => 'rare_drop'
# ... no more has_many declarations because this is a small table and this is it!
end</code></pre>Not much more to say about this code. It's pretty standard by now. As for the d<span style="font-family: courier;">b/seeds.rb</span> script, it's an easy addition of two lines at the end of the loop that creates all of the monsters:
<pre><code class="ruby">csv_file_path = 'db/monsters.csv'
CSV.foreach(csv_file_path, {headers: true}) do |row|
# ... all of the other monster attribute processing code ...
monster['normal_drop'] = Material.find_by(name: monster['normal_drop'])
monster['rare_drop'] = Material.find_by(name: monster['rare_drop'])
Monster.create!(monster)
puts "#{row['name']} added!"
end</code></pre>Easy as pie. This code even takes care of all of those drops that are not monster materials because it won't find them in the Material table and just assigns the monster attributes to nil. It's so elegant and simple, just how I like my code. At this point we're ready to wipe it and redo it at the command prompt:
<pre><code class="csh">$ rails db:drop
$ rails db:migrate
$ rails db:seed</code></pre>When those commands are done running, we're almost ready to see what we've done.<div><br /></div><div><h4 style="text-align: left;">Adding Materials to the Monster View</h4></div><div>This section will be quick, as we only need to add a couple of table entries to the <a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html" target="_blank">monster detail view</a> in <span style="font-family: courier;">app/views/monster/show.html.erb</span>:</div><pre><code class="html"><h1><%= @monster.name %></h1>
<table id="monster-table" class="table table-striped table-sm">
<!--many table rows-->
<tr>
<td><strong>Halved for</strong></td>
<td><%= @monster.halved %></td>
<td><strong>Resistant to</strong></td>
<td><%= @monster.resistant %></td>
</tr>
<tr>
<td><strong>Normal Drop</strong></td>
<td>
<% if @monster.normal_drop %>
<%= @monster.normal_drop.name %>
<% else %>
--
<% end %>
</td>
<td><strong>Rare Drop</strong></td>
<td>
<% if @monster.rare_drop %>
<%= @monster.rare_drop.name %>
<% else %>
--
<% end %>
</td>
</tr>
<!--many more table rows-->
</table></code></pre>
We'll add the drops right after the halved and resistant attributes, and we can see that all of our script changes worked:<div><br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRHufWqB497NlLGa_NF02aCrjgiskiUYLJHlktj_mBjlNVNJT-5U3Q6qIPBy687JYK75D4kweqrTyUECojvxH7AggGkTnTVEFCReOi_mKqaMKuL_tJHtS8wzsWKEJ3Rn6tLNhhqZ_85lY/s1255/Screenshot_narasimha_with_drops.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of Narasimha monster with drop attributes" border="0" data-original-height="614" data-original-width="1255" height="196" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRHufWqB497NlLGa_NF02aCrjgiskiUYLJHlktj_mBjlNVNJT-5U3Q6qIPBy687JYK75D4kweqrTyUECojvxH7AggGkTnTVEFCReOi_mKqaMKuL_tJHtS8wzsWKEJ3Rn6tLNhhqZ_85lY/w400-h196/Screenshot_narasimha_with_drops.png" width="400" /></a></div><br /><h4 style="text-align: left;">Filtering Monsters on Materials</h4><div>Before we go, let's bring this exercise to its logical conclusion by adding links from the monster details page to the materials page, highlighting the material that was clicked, and add links to the materials page that go back to a <a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html" target="_blank">filtered list of monsters</a> that drop that material sorted by location depth. That way we can easily find the first monsters that drop a given material. </div><div><br /></div><div>First, we can edit the table row we just added to <span style="font-family: courier;">app/views/monster/show.html.erb</span> to add a link with a <span style="font-family: courier;">:highlight</span> tag to the monster material names:</div>
<pre><code class="html"><h1><%= @monster.name %></h1>
<table id="monster-table" class="table table-striped table-sm">
<!--many table rows-->
<tr>
<td><strong>Halved for</strong></td>
<td><%= @monster.halved %></td>
<td><strong>Resistant to</strong></td>
<td><%= @monster.resistant %></td>
</tr>
<tr>
<td><strong>Normal Drop</strong></td>
<td>
<% if @monster.normal_drop %>
<%= link_to @monster.normal_drop.name,
material_index_path(highlight: @monster.normal_drop.name) %>
<% else %>
--
<% end %>
</td>
<td><strong>Rare Drop</strong></td>
<td>
<% if @monster.rare_drop %>
<%= link_to @monster.rare_drop.name,
material_index_path(highlight: @monster.rare_drop.name) %>
<% else %>
--
<% end %>
</td>
</tr>
<!--many more table rows-->
</table></code></pre>
The code is now littered with these features, so it's pretty easy to copy other examples in the code to make these changes work. The mechanics are all explained in detail in the <a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html" target="_blank">Viewing the Monster Data episode</a>. The next change is to assign the <span style="font-family: courier;">:highlight</span> parameter that was passed in the URL to a variable in <span style="font-family: courier;">app/controllers/material_controller.rb</span>:
<pre><code class="ruby">class MaterialController < ApplicationController
def index
@materials = Material.all
@highlighted = params[:highlight]
end
end</code></pre>
<div>Then we can use the <span style="font-family: courier;">@highlight</span> variable in <span style="font-family: courier;">app/views/material/index.html.erb</span>:
<pre><code class="html"><h1>Monster Materials</h1>
<table id="material-table" class="table table-striped table-sm">
<thead class="thead-dark">
<tr>
<th scope="col">Name</th>
<th scope="col">Grade</th>
<th scope="col">Type</th>
</tr>
</thead>
<% @materials.each do |material| %>
<tr <%= "class=table-primary" if material.name == @highlighted %>>
<td><%= link_to material.name,
monster_index_path(:drop_filter => material.name) %></td>
<td><%= material.grade %></td>
<td><%= material.material_type %></td>
</tr>
<% end %>
</table></code></pre>
Like in the other tables, we simply add a class of <span style="font-family: courier;">table-primary</span> to the highlighted row. I also went ahead and added the link back to the monster table with a <span style="font-family: courier;">drop_filter</span> parameter. To use the filter parameter, we can modify <span style="font-family: courier;">app/controllers/monster_controller.rb</span>:
<pre><code class="ruby">class MonsterController < ApplicationController
def index
if params[:filter]
location = Location.find_by(name: params[:filter])
@monsters = location.location_monsters +
location.location2_monsters +
location.location3_monsters
elsif params[:ability_filter]
ability = Ability.find_by(name: params[:ability_filter])
@monsters = ability.get_all_monsters.sort_by { |monster| monster.first_location_depth }
elsif params[:skill_filter]
ability = RoleAbility.find_by(name: params[:skill_filter])
@monsters = ability.get_all_monsters.sort_by { |monster| monster.first_location_depth }
elsif params[:drop_filter]
material = Material.find_by(name: params[:drop_filter])
@monsters = material.get_all_monsters.sort_by { |monster| monster.first_location_depth }
else
@monsters = Monster.all
end
end
def show
@monster = Monster.find(params[:id])
end
end</code></pre>
Finally, we need to define the super simple <span style="font-family: courier;">get_all_monsters</span> for the Material class in <span style="font-family: courier;">app/models/material.rb</span>:
<pre><code class="ruby">class Material < ApplicationRecord
has_many :normal_drop_monsters, :class_name => 'Monster', :foreign_key => 'normal_drop'
has_many :rare_drop_monsters, :class_name => 'Monster', :foreign_key => 'rare_drop'
def get_all_monsters()
normal_drop_monsters + rare_drop_monsters
end
end</code></pre>
Amazingly, with this change we are done, and we have all of the functionality we were aiming for with the monster materials table. Here's a view of monsters that drop Potent Sliver and are sorted by location depth:<div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3RfRo_CKbR9jxlpmykWw9lWFFrrH_ES3Nuyf1YYh-09wXxtp9-phUKGc1C0Pk8QVJ-kSHYkehorZNcnij7DbL20EqKp8NpN6ABRufCQFXelrWtgFqEbfdNJxiAWe_CwioB3PBk6qBcYc/s1235/Screenshot_monster_filtered_by_potent_sliver.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of monsters filtered by potent sliver drop" border="0" data-original-height="666" data-original-width="1235" height="216" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3RfRo_CKbR9jxlpmykWw9lWFFrrH_ES3Nuyf1YYh-09wXxtp9-phUKGc1C0Pk8QVJ-kSHYkehorZNcnij7DbL20EqKp8NpN6ABRufCQFXelrWtgFqEbfdNJxiAWe_CwioB3PBk6qBcYc/w400-h216/Screenshot_monster_filtered_by_potent_sliver.png" width="400" /></a></div><br /><div>We should take a moment to appreciate all that we've accomplished here. In one sweep of the code we've parsed out the normal and rare drops for all monsters; merged those dropped items with the existing list of monsters; added the new drop attributes to the database and models; seeded the database with the material drops; connected the monster and material models together by the materials in common; linked the monster detail and material views together; and showed a filtered and sorted list of monsters that drop a selected material. That's basically a full review of everything we've been doing in this series, all wrapped into one exercise, and it wasn't all that much code! Next time, we'll finally take care of the last straggling table of monster characteristics and do something a little different with it.</div></div></div></div><div><br /></div><div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div></div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-35718044048474927752020-08-24T17:39:00.002-05:002020-10-26T20:20:06.514-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2: Asking Deeper QuestionsSo far in this <a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Exploring Monster Taming Mechanics series</a>, we've parsed a bunch of data, built up database tables, and connected them together in a website with some <a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">filtering sprinkled in</a>. This setup allows us to browse around the data pretty easily and ask some basic questions of the data. Now it's time to think about how we can ask deeper questions. Instead of just asking things like, "What monsters have the Auto-Bravery ability," we want to be able to ask, "Where is the earliest location where I can get a monster with the Auto-Bravery ability?" Sounds like a useful think to know, right? Let's figure it out.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2Fs4Fe33T5VMSNYPALVGnNZ_C_P6LnFto1eX-TGHjFPgEeXys7zzgO-mfTNmuEJNNbYF9pqyZpcSQ3tzzSMkyRziZ1uCvQuUDnEtZnN8DgcmApostTTdJHXMlrXe5iACmzZl1UN9TO2M/s1280/FFXIII-2-Narasimha.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Final Fantasy XIII-2 Narasimha tamed" border="0" data-original-height="720" data-original-width="1280" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2Fs4Fe33T5VMSNYPALVGnNZ_C_P6LnFto1eX-TGHjFPgEeXys7zzgO-mfTNmuEJNNbYF9pqyZpcSQ3tzzSMkyRziZ1uCvQuUDnEtZnN8DgcmApostTTdJHXMlrXe5iACmzZl1UN9TO2M/w400-h225/FFXIII-2-Narasimha.jpg" width="400" /></a></div><br />
<a name='more'></a><h4>
Are we asking the right question?</h4>
<script>hljs.initHighlightingOnLoad();</script>
It would certainly be interesting to know the earliest location where we can get a monster with a given ability, but we have access to more data than that, so we could actually answer a bigger question with the same effort. How about, "what are the locations where I can get a monster with the Auto-Bravery ability, sorted from earliest to latest?" We still need to find the earliest location where a tamable monster has the Auto-Bravery ability, but we can find out more this way. We've transformed the question from a linear search through the locations from earliest to latest into a filter-and-sort query, which is easy for a database to do.<div><br /></div><div>This filter-and-sort query is still more complex than previous questions because we're taking into account more types of data. Instead of just filtering monsters by ability, we also need to sort them by location, and to do that sorting, we need to know the order of the locations. We do have a complete ordering in the database table because the locations were added to the table in such a way that dependent locations were added after their source locations were added. </div><div><br /></div><div>That's not the only valid ordering of the locations, though. Sometimes the game path forks, and in those cases the locations that both have the same source can be reached in either order. Since the locations make a tree graph with New Bodhum 003 AF at the root, we can add a depth attribute to the location table and assign each location a depth value based on how far it is away from the root. This depth value represents how many areas must be visited if we headed straight for the area in question. We could do this algorithmically, but the table only has 30 locations, so it's easy enough to assign by hand. We've added attributes to tables and views a half dozen times now, so I'll assume it's obvious how to do this and move on.</div><div><br /></div><h4 style="text-align: left;">Sorting by Location Depth</h4><div>Sorting by location depth isn't as hard as it may appear. We already have the filtering done between the links in the location table and the filter parameter processing in the monster controller, and we have the filtered list of monsters in the monster controller. All we have to do is sort that list by the location depth for where we can find each monster. Also, remember that a monster can be found in up to three different locations, so we're going to have to handle that detail as well. First, let's go ahead and add the sorting by location depth to the monster controller in <font face="courier">app/controllers/monster_controller.rb</font>:</div>
<pre><code class="ruby">class MonsterController < ApplicationController
def index
if params[:filter]
location = Location.find_by(name: params[:filter])
@monsters = location.location_monsters +
location.location2_monsters +
location.location3_monsters
elsif params[:ability_filter]
ability = Ability.find_by(name: params[:ability_filter])
@monsters = ability.get_all_monsters.sort_by { |monster| monster.first_location_depth }
elsif params[:skill_filter]
ability = RoleAbility.find_by(name: params[:skill_filter])
@monsters = ability.get_all_monsters.sort_by { |monster| monster.first_location_depth }
else
@monsters = Monster.all
end
end
def show
@monster = Monster.find(params[:id])
end
end</code></pre>It ends up reading in code just about the same as it reads in the above description. I went ahead and did the sort on both abilities and skills here because it's the same code. Now, we need to define this <font face="courier">first_location_depth</font> method that we've called on the monster model. As long as it returns the depth of the location where the monster can first be found, <font face="courier">sort_by</font> will end up sorting the list of monsters the way we want it to. Here's one way to define that method in <font face="courier">app/models/monster.rb</font>:<br />
<pre><code class="ruby">class Monster < ApplicationRecord
# ... A whole mess of belongs_to macros ...
def first_location_depth
[location&.depth, location2&.depth, location3&.depth].compact.min
end
end</code></pre>We simply build up a short array of depths using the existence (&.) operator for each potential location, compact the array to remove the nil values for the locations that didn't exist, and return the minimum value. That's all there is to it. When we click on the Auto-Bravery ability in the ability table, we are presented with a list of monsters with that ability sorted by the depth of the first location where we can find them:<div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUMwkyfOyKviFPaXEpCmGsyutTVVolyi39cKL9GgDrP6CVG7Ep7TWsmnfzGHVYUdMaH9GGER5s19AZ3MnRiOa6f5jSZL2CDq0dqqB7ZBaQn2HzGOTQCYnmB6tWSqof6Y4CB5E52kQIuL4/s1304/monsters_auto-bravery_by_depth_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of list of monsters with Auto-Bravery sorted by first location depth" border="0" data-original-height="597" data-original-width="1304" height="229" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUMwkyfOyKviFPaXEpCmGsyutTVVolyi39cKL9GgDrP6CVG7Ep7TWsmnfzGHVYUdMaH9GGER5s19AZ3MnRiOa6f5jSZL2CDq0dqqB7ZBaQn2HzGOTQCYnmB6tWSqof6Y4CB5E52kQIuL4/w500-h229/monsters_auto-bravery_by_depth_screenshot.png" width="500" /></a></div><br /><div><h4 style="text-align: left;">What's Left?</h4></div>
This feature of filtering and sorting by first location depth is pretty useful for finding monsters with certain abilities to tame. It's so useful, I wanted to do something similar with monsters' base strength or max level or some combination of those, but I realized that there's already a pretty effective way of looking for the strongest monsters at a given point in the game just by filtering monsters by location. You're normally interested in seeing which monsters you want to tame and keep in the location where you are right now or heading to next, and it's simple enough to just find the strongest monsters that can be leveled up the most by visual inspection of the filtered monster table. There aren't enough monsters in any given location to make this method burdensome, and we have plenty of info in this table to make informed choices.</div><div><br /></div><div>Trying to automate this question of finding the strongest monsters also adds its own complications. What should we sort by, base strength, max level, or some combination of them? What if it's not what the user wants? Should we give the user the option? The solution to these questions will necessarily add complexity to the user interface that we should prefer to avoid. Side-stepping the issue and just not trying to do something for the user that is easier for them to do for themselves ends up being the better solution. We should always watch out for that simple case when designing a user interface.</div><div><br />Since we already have the feature of finding strong monsters in each location, the only things remaining are the two tables that we haven't connected: monster materials and monster characteristics. Monster materials is an interesting table that we can use to figure out how quickly we can level up the monsters we've tamed because we need to find the monsters that drop those materials that are used to level up our tamed monsters. However, enabling that feature is going to take a bit of work because the material table doesn't currently have the info for which monsters drop each material. We'll add that missing data and connect up that table next time.<br />
</div><div><br /></div><div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div></div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-24921710736150275022020-08-03T20:45:00.001-05:002020-10-26T20:19:56.716-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2: Filtering MonstersWe've finished <a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html" target="_blank">building up views and controllers in the Rails MVC architecture</a> for the various tables of monster data we have in this Final Fantasy XIII-2 monster taming mechanics series. However, some of the navigation for the latest monster details page and ability table pages is a bit clunky, so we're going to improve that today. First, when a link is clicked in the monster details page to go to the ability table, the clicked ability is highlighted, but the user needs to search through the table to find the highlighted row. We'll see how to go to the desired row directly. Then, we want to make the return trip from a link in the ability table take the user to a monster table that shows only the monsters that have that ability. Let's see how it's done.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRuRDS_pXY5FTXrp9s8sbEayGpH4dEwP7fUrXhmQyB3kibavXTZewzmgM1qpj5aAZkc4LA6alAIFSVt7_KZU_ljJu1O6ZDbYHAoCS_4GINLFogHdfi802XYoLBofJCCzY8IcdsNGHe830/s1600/FFXIII-2-Battle-Scene-4.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Final Fantasy XIII-2 Battle Scene" border="0" data-original-height="349" data-original-width="620" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRuRDS_pXY5FTXrp9s8sbEayGpH4dEwP7fUrXhmQyB3kibavXTZewzmgM1qpj5aAZkc4LA6alAIFSVt7_KZU_ljJu1O6ZDbYHAoCS_4GINLFogHdfi802XYoLBofJCCzY8IcdsNGHe830/s400/FFXIII-2-Battle-Scene-4.jpg" title="" width="400" /></a></div>
<br />
<a name='more'></a><h4>
Jump to a Table Row</h4>
<script>hljs.initHighlightingOnLoad();</script>
It's not a pleasant experience to click on a link in the monster details page that takes us to one of the ability tables, and then have to search through that long table for the highlighted link. It would be much nicer if the link could take us to the highlighted row automagically. Luckily, that's not too hard to do with HTML anchors. In order to set up this page jump, we can first add an anchor to the default passive ability links in the monster detail page. This anchor name needs to be a string with no spaces, so we'll create a string like "ability100" using the ability's database ID. This code is near the end of <span style="font-family: "courier new" , "courier" , monospace;">app/views/monster/show.html.erb</span>:<br />
<pre><code class="html"> <td><strong>Default</strong></td>
<td>
<% (1..4).each do |n| %>
<% ability = @monster.send "default_passive#{n}" %>
<% if ability %>
<%= link_to ability.name,
ability_index_path(anchor: "ability#{ability.id}", highlight: ability.name) %><br />
<% end %>
<% end %>
</td></code></pre>
Note that I added a variable <span style="font-family: "courier new" , "courier" , monospace;">ability</span> to make the rest of the code more concise, and so we don't need to send the default passive string to the monster model over and over again. Otherwise, the only change was to add an <span style="font-family: "courier new" , "courier" , monospace;">:anchor</span> value to the parameters sent to the <span style="font-family: "courier new" , "courier" , monospace;">ability_index_path</span> helper function. This <span style="font-family: "courier new" , "courier" , monospace;">:anchor</span> will end up as a hashtag at the end of the URL, and if we have a target that corresponds to that hashtag on the page that we're going to, the browser will jump down the page to that tag. To add the appropriate tags on the passive ability page, we simply add anchor tags (<a>) with id strings to each row in <span style="font-family: "courier new" , "courier" , monospace;">app/views/ability/index.html.erb</span>:<br />
<pre><code class="html"> <% @abilities.each do |ability| %>
<tr <%= "class=table-primary" if ability.name == @highlighted %>>
<td><a id=<%= "ability#{ability.id}" %>><%= ability.name %></a></td>
<td><%= ability.rank %></td>
<td><%= ability.description %></td>
</tr>
<% end %>
</table></code></pre>
With that done, we can test out our new feature by clicking on a default passive ability in any monster detail page:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj57ZiicGJ7BZ_mgQBfjxGxxMFeYvjNyw1789B2Vl2GHBf9pbnNut9LdpBBhB7gssu2AIfIGqm_qmu22iIcu7h617ckWt1fIb5FHGHF0rnEEAHhAavBbab7Y1Yy_P0aOgJkZ2JMQsfbXFg/s1600/monster_ability_jump_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of monster ability table after jump to ability" border="0" data-original-height="547" data-original-width="1282" height="170" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj57ZiicGJ7BZ_mgQBfjxGxxMFeYvjNyw1789B2Vl2GHBf9pbnNut9LdpBBhB7gssu2AIfIGqm_qmu22iIcu7h617ckWt1fIb5FHGHF0rnEEAHhAavBbab7Y1Yy_P0aOgJkZ2JMQsfbXFg/s400/monster_ability_jump_screenshot.png" title="" width="400" /></a></div>
Huh, that's funny. We definitely jumped further down the table, but where's the highlighted row? It turns out that it's behind the page header, and if we scroll up a bit, we'll find it right there. That's not the greatest experience in the world either, but it can be fixed with a little trick. All we have to do is increment the anchor ids in the table a few times, and we'll jump to the ability that many before the one we want:<br />
<pre><code class="html"> <% @abilities.each do |ability| %>
<tr <%= "class=table-primary" if ability.name == @highlighted %>>
<td><a id=<%= "ability#{ability.id + 3}" %>><%= ability.name %></a></td>
<td><%= ability.rank %></td>
<td><%= ability.description %></td>
</tr>
<% end %>
</table></code></pre>
Now we can see the highlighted row after the jump:
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDR7PUw82SqboCEItV92TaelQkmbvTS11TGPeAptK1CgqO5LJFzxaIQFz2iHqZc3Pc5C5pszHjaQDF6Ud8XTjNedddd8WsI-g8vGbFrPNV81rjbVXaKhwk0I_1YDcH7wIGJyo83BuG3Cw/s1600/monster_ability_jump_further_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of monster ability table after jumping a little past the ability" border="0" data-original-height="580" data-original-width="1290" height="178" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDR7PUw82SqboCEItV92TaelQkmbvTS11TGPeAptK1CgqO5LJFzxaIQFz2iHqZc3Pc5C5pszHjaQDF6Ud8XTjNedddd8WsI-g8vGbFrPNV81rjbVXaKhwk0I_1YDcH7wIGJyo83BuG3Cw/s400/monster_ability_jump_further_screenshot.png" title="" width="400" /></a></div>
<br />
So what happens if we try to jump to "ability1" now, which no longer exists? It turns out that's just fine because the browser will stay at the top of the page, and the highlighted first row will be visible anyway! Since the feature is working, we can quickly add anchors to the other abilities, like the default skills in <span style="font-family: "courier new" , "courier" , monospace;">app/views/monster/show.html.erb</span>:<br />
<pre><code class="html"> <td>
<% (1..8).each do |n| %>
<% skill = @monster.send "default_skill#{n}" %>
<% if skill %>
<%= link_to skill.name,
role_ability_index_path(anchor: "skill#{skill.id}", highlight: skill.name) %><br />
<% end %>
<% end %>
</td>
</code></pre>
And the rest of the level abilities and skills in <span style="font-family: "courier new" , "courier" , monospace;">app/views/monster/_level_ability.html.erb</span>:<br />
<pre><code class="html"> <% if @monster.has_ability?(passive) || @monster.has_ability?(skill) %>
<tr>
<td><strong>Level <%= "%02d" % [n] %></strong></td>
<td>
<% if @monster.has_ability? passive %>
<% ability = @monster.send passive %>
<%= link_to ability.name,
ability_index_path(anchor: "ability#{ability.id}", highlight: ability.name) %><br />
<% end %>
</td>
<td/>
<td>
<% if @monster.has_ability? skill %>
<% ability = @monster.send skill %>
<%= link_to ability.name,
role_ability_index_path(anchor: "skill#{ability.id}", highlight: ability.name) %><br />
<% end %>
</td>
</tr>
<% end %></code></pre>
Finally, we need to add anchor tags to the role ability (skill) table in <span style="font-family: "courier new" , "courier" , monospace;">app/views/role_ability/index.html.erb</span>:<br />
<pre><code class="html"> <% @role_abilities.each do |ability| %>
<tr <%= "class=table-primary" if ability.name == @highlighted %>>
<td><a id=<%= "skill#{ability.id+3}" %>><%= ability.name %></a></td>
<td><%= ability.role %></td>
<td><%= ability.infusable %></td>
</tr>
<% end %>
</table></code></pre>
And with that, we have page jumps working for both ability tables. Notice that this was all done in the HTML template files. We didn't even have to touch the controllers or the models to make this feature happen.<br />
<br />
<h4>
Linking From Abilities to Monsters</h4>
<div>
The other feature we want to add to this website is a link back from the ability tables to the monster table, and in following these links, we want to filter the monster table down to only those monsters that have the selected ability. The tricky part about this feature is that the abilities can show up in any of the default ability or level ability properties for each monster, so we're going to have to be a bit more careful to do a complete search. The first thing we want to do, though, is create the ability links and add the filter to the URL when the links are clicked. We'll start with the passive ability table in <span style="font-family: "courier new" , "courier" , monospace;">app/views/ability/index.html.erb</span>:</div>
<pre><code class="html"> <% @abilities.each do |ability| %>
<tr <%= "class=table-primary" if ability.name == @highlighted %>>
<td>
<a id=<%= "ability#{ability.id+3}" %>>
<%= link_to ability.name, monster_index_path(ability_filter: ability.name) %>
</a>
</td>
<td><%= ability.rank %></td>
<td><%= ability.description %></td>
</tr>
<% end %>
</table></code></pre>
It's the same kind of link with a filter that we made when <a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html" target="_blank">filtering by location</a>, but since we already have that location filter, we want to name this filter something different so they won't conflict. Hence, <span style="font-family: "courier new" , "courier" , monospace;">ability_filter</span>. The only other thing we need to do is use the filter in the controller, but we need to get creative about finding all of the monsters that have the ability in the filter so that we don't have to list all 100 levels of potential abilities like we did with locations. Here's one way to do it in <span style="font-family: "courier new" , "courier" , monospace;">app/controllers/monster_controller.rb</span>:<br />
<pre><code class="ruby">class MonsterController < ApplicationController
def index
if params[:filter]
location = Location.find_by(name: params[:filter])
@monsters = location.location_monsters +
location.location2_monsters +
location.location3_monsters
elsif params[:ability_filter]
ability = Ability.find_by(name: params[:ability_filter])
@monsters = ability.get_all_monsters
else
@monsters = Monster.all
end
end
def show
@monster = Monster.find(params[:id])
end
end</code></pre>
The <span style="font-family: "courier new" , "courier" , monospace;">:ability_filter</span> search has the same form as the <span style="font-family: "courier new" , "courier" , monospace;">:filter</span> search, but what does this <span style="font-family: "courier new" , "courier" , monospace;">ability.get_all_monsters</span> method do? All of the work for finding the monsters that have the specified ability is done in this method so that the logic that searches the model doesn't clutter up the controller. Instead, it's tucked away in the Ability model in <span style="font-family: "courier new" , "courier" , monospace;">app/models/ability.rb</span>:<br />
<pre><code class="ruby">class Ability < ApplicationRecord
# ... a whole mess of has_many macros ...
def get_lv_monsters(n)
lv_nn_passive_monsters = "lv_%02d_passive_monsters" % [n]
respond_to?(lv_nn_passive_monsters) && send(lv_nn_passive_monsters) || []
end
def get_all_monsters()
(1..4).map { |n| send "default_passive#{n}_monsters" }.flatten +
(2..99).map{ |n| get_lv_monsters(n) }.flatten
end
end</code></pre>
We can see that the <span style="font-family: "courier new" , "courier" , monospace;">get_all_monsters</span> method collects the monsters for the four default passive ability attributes and then does something similar for the 99 level passive ability attributes using the <span style="font-family: "courier new" , "courier" , monospace;">get_lv_monsters(n)</span> method. Sending the message of "<span style="font-family: "courier new" , "courier" , monospace;">default_passive#{n}_monsters</span>" with n = 1..4 uses the <span style="font-family: "courier new" , "courier" , monospace;">has_many</span> mechanisms set up in the ability model. Each <span style="font-family: "courier new" , "courier" , monospace;">has_many</span> macro sets up a reference to each monster that has the selected ability as its first through fourth default passive abilities. This is where those macros come in handy, like they did when setting up the location filters before.<br />
<br />
The search for the monsters that have passive abilities in the other levels has to be more careful because some level ability attributes don't exist at all. First, we use <span style="font-family: "courier new" , "courier" , monospace;">respond_to?</span> to check that the method name exists. If it does, we move to the second part of the <span style="font-family: "courier new" , "courier" , monospace;">&&</span> operator and send the message to get the monsters with that level n ability. If there are any monsters, the list gets returned right away, but if there are no monsters with that level n ability, false would get returned. We don't want that, so we tack an "<span style="font-family: "courier new" , "courier" , monospace;">|| []</span>" on the end of the line to return an empty list instead. All of these potential non-empty lists of monsters gets concatenated with <span style="font-family: "courier new" , "courier" , monospace;">map</span> and then flattened.<br />
<br />
We're now ready to test out this feature, and if we follow the first monster, Apkallu, to its details page and then click on the first default passive ability, we get a table of monsters that have the "Attack: ATB Charge" ability:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAsaiwdw5wlhhzRgwIa59O2EHbDYxQGrVUHMPb8BlDaQeoajWiQvN7bAEJWpUbeUWp78d44h3hUOHla5wwHaUgSG19kGoMO1WIi8ghrtNoEjq2f_eA-djjcelKaTA_N7_gRwfUxjSns0E/s1600/monster_ability_filter_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of monster table filtered by ability" border="0" data-original-height="589" data-original-width="1303" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAsaiwdw5wlhhzRgwIa59O2EHbDYxQGrVUHMPb8BlDaQeoajWiQvN7bAEJWpUbeUWp78d44h3hUOHla5wwHaUgSG19kGoMO1WIi8ghrtNoEjq2f_eA-djjcelKaTA_N7_gRwfUxjSns0E/s400/monster_ability_filter_screenshot.png" title="" width="400" /></a></div>
<br />
This table also happens to be sorted by where the ability appears in the default or level abilities because of how we constructed the list, so if you clicked on the Chichu at the end, you would see the "Attack: ATB Charge" show up as a level 15 ability, whereas the rest of the monsters have it as a default ability. That might be a nice thing to see in this table, but we won't try to figure out how to add that information right now. Instead, let's finish up by doing the same filtering for skills.<br />
<br />
First, we add the links and filters to <span style="font-family: "courier new" , "courier" , monospace;">app/views/role_ability/index.html.erb</span>:<br />
<pre><code class="html"> <% @role_abilities.each do |ability| %>
<tr <%= "class=table-primary" if ability.name == @highlighted %>>
<td>
<a id=<%= "ability#{ability.id+3}" %>>
<%= link_to ability.name, monster_index_path(skill_filter: ability.name) %>
</a>
</td>
<td><%= ability.role %></td>
<td><%= ability.infusable %></td>
</tr>
<% end %>
</table></code></pre>
There's nothing new here. We're just copying what we did with the ability view and making sure we rename the filter to <span style="font-family: "courier new" , "courier" , monospace;">skill_filter</span>. Then, we can add the call to the Role Ability model in <span style="font-family: "courier new" , "courier" , monospace;">app/controllers/monster_controller.rb</span>:<br />
<pre><code class="ruby">class MonsterController < ApplicationController
def index
if params[:filter]
location = Location.find_by(name: params[:filter])
@monsters = location.location_monsters +
location.location2_monsters +
location.location3_monsters
elsif params[:ability_filter]
ability = Ability.find_by(name: params[:ability_filter])
@monsters = ability.get_all_monsters
elsif params[:skill_filter]
ability = RoleAbility.find_by(name: params[:skill_filter])
@monsters = ability.get_all_monsters
else
@monsters = Monster.all
end
end
def show
@monster = Monster.find(params[:id])
end
end</code></pre>
Finally, we can add similar methods to the Role Ability model, taking care to make the adjustments for the differences in this model, in <span style="font-family: "courier new" , "courier" , monospace;">app/models/role_ability.rb</span>:<br />
<pre><code class="ruby">class RoleAbility < ApplicationRecord
# ... that mess of has_many macros again ...
def get_lv_monsters(n)
lv_nn_skill_monsters = "lv_%02d_skill_monsters" % [n]
respond_to?(lv_nn_skill_monsters) && send(lv_nn_skill_monsters) || []
end
def get_all_monsters()
(1..8).map { |n| send "default_skill#{n}_monsters" }.flatten +
(2..99).map{ |n| get_lv_monsters(n) }.flatten
end
end</code></pre>
And we've completed the filter for role abilities, a.k.a. skills. We can test it out by clicking on an ability link in the role abilities table, and see that yes, the monster table is filtered down to the monsters that have that skill, in this case adrenaline:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBFIkSCIiAnbhT9aDDv3Nietg7mpHWIjyVZNNKvoXbb3jr0dkvlAN0QdKNRqsJXtB6TZPCsaX6bB2M01wVxBvrj9fAmj3HChJvzl9zCsxvd4lzXNfmeHNxfnOXLSmk40NPG_DB_jHmP4g/s1600/monster_skill_filter_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of monster table filtered by adrenaline skill" border="0" data-original-height="613" data-original-width="1288" height="190" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBFIkSCIiAnbhT9aDDv3Nietg7mpHWIjyVZNNKvoXbb3jr0dkvlAN0QdKNRqsJXtB6TZPCsaX6bB2M01wVxBvrj9fAmj3HChJvzl9zCsxvd4lzXNfmeHNxfnOXLSmk40NPG_DB_jHmP4g/s400/monster_skill_filter_screenshot.png" title="" width="400" /></a></div>
<br />
At this point we're reaching a fairly complete website with this collection of tables that are connected together in important ways. There are a couple problematic tables, like the materials table isn't really connected to anything and the characteristics table is even more troublesome with monster characteristics showing up in multiple different monster properties. But overall, things are looking pretty fleshed out. We can get answers to quite a few questions now just by clicking around the tables and looking at the filtered monster table like, "Which monsters are in Bresha Ruins 100 AF?" or "Which monsters have the Critical: Haste passive ability, and where can I find them?" It's time to start thinking about how to answer some harder questions like, "What is the earliest place where I can get a monster with the Auto-Bravery passive ability?" We'll try to tackle that more complex beast next time.<div><br /></div><div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div></div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-15522992530674674672020-07-13T20:44:00.001-05:002020-10-26T20:19:45.791-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2: Viewing More Monster Data and AbilitiesWe've been building up <a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html" target="_blank">views and controllers in the Rails MVC architecture</a> for the various tables of monster data we have in this Final Fantasy XIII-2 monster taming mechanics series. We're now almost ready to finish off these table views with the ability tables, but before we do that, we need a place for the links to those ability tables to exist. The number of abilities for each monster is too much to put in the main monster table page, so we'll need to first build a details page for each monster before we can link up the monster abilities with the ability tables. Let's get started on that details page.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7eSErjZLwOecC_Ugzb7-YaaxcZwt6dJnsEWcnui8FbA6a3ZE15BDZdDWqopw2iGzEEa190ZE_5qcoFGR2GQ7lUbZXViFIDQFQZQWr4varIwyQJB-W3moEKcFEsJzFz4koi-hT4Fz3FtE/s1600/FFXIII-2-Battle-Scene-3.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Final Fantasy XIII-2 Battle Scene" border="0" data-original-height="416" data-original-width="740" height="223" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7eSErjZLwOecC_Ugzb7-YaaxcZwt6dJnsEWcnui8FbA6a3ZE15BDZdDWqopw2iGzEEa190ZE_5qcoFGR2GQ7lUbZXViFIDQFQZQWr4varIwyQJB-W3moEKcFEsJzFz4koi-hT4Fz3FtE/s400/FFXIII-2-Battle-Scene-3.jpg" title="" width="400" /></a></div>
<br />
<a name='more'></a><h4>
Create a Monster Details Page</h4>
<script>hljs.initHighlightingOnLoad();</script>
Creating a monster details page should be about as straightforward as creating the table pages. We simply need to use a different action in the monster controller to show a page for a single monster instead of the index action that shows a table of all monsters. We can't use a Rails generator for this task because we already used one to create the monster index action, and we can't add to that same controller and view with a generator. We can only create new controllers and views with a generator; hence, the name. That's okay, though, because all we have to do is add to a few files to make this page happen. First, we need to add a route in <span style="font-family: "courier new" , "courier" , monospace;">config/routes.rb</span>. Instead of adding another <span style="font-family: "courier new" , "courier" , monospace;">get</span> route, we can use the <span style="font-family: "courier new" , "courier" , monospace;">resource</span> method to combine the index and show routes for the monster pages:<br />
<pre><code class="ruby">Rails.application.routes.draw do
resources :monster, only: [:index, :show]
get 'location/index'
get 'characteristic/index'
get 'material/index'
get 'home/index'
root 'home#index'
end</code></pre>
Using the <span style="font-family: "courier new" , "courier" , monospace;">resource</span> method will create the routes a bit differently, so the monster index will live at <span style="font-family: "courier new" , "courier" , monospace;">localhost:3000/monster</span> instead of <span style="font-family: "courier new" , "courier" , monospace;">localhost:3000/monster/index</span>, and the individual monster pages will live at <span style="font-family: "courier new" , "courier" , monospace;">localhost:3000/monster/:id</span>, where :id is the database index of the monster we're looking at. All of the links we've created will conveniently change to the new form automatically because we used the <span style="font-family: "courier new" , "courier" , monospace;">monster_index_path</span> helper method to set the destination of the links to the monster index page.<br />
<br />
Now that we have a new route for individual monster pages, we can create a new action in <span style="font-family: "courier new" , "courier" , monospace;">app/controllers/monster_controller.rb</span>:<br />
<pre><code class="ruby"> def show
@monster = Monster.find(params[:id])
end</code></pre>
Preparing the data for the view is as simple as looking up the :id in the model that was provided in the URL. We'll see how we set this :id in the URL in a bit, but first, let's build the monster detail page in <span style="font-family: "courier new" , "courier" , monospace;">app/views/monster/show.html.erb</span> using a two-column table layout:<br />
<pre><code class="html"><h1><%= @monster.name %></h1>
<table id="monster-table" class="table table-striped table-sm">
<tr>
<td><strong>Role</strong></td>
<td><%= @monster.role %></td>
<td/>
<td/>
</tr>
<tr>
<td><strong>Monster Type</strong></td>
<td><%= @monster.monster_type %></td>
<td><strong>Constellation</strong></td>
<td><%= @monster.constellation %></td>
</tr>
<tr>
<td><strong>Location</strong></td>
<td>
<%= link_to @monster.location.name, location_index_path(:highlight => @monster.location.name) %>
<% if @monster.location2 %>
<br /><%= link_to @monster.location2.name, location_index_path(:highlight => @monster.location2.name) %>
<% end %>
<% if @monster.location3 %>
<br /><%= link_to @monster.location3.name, location_index_path(:highlight => @monster.location3.name) %>
<% end %>
</td>
<td><strong>Tame Rate</strong></td>
<td><%= @monster.tame_rate %></td>
</tr>
<tr>
<td><strong>Max Level</strong></td>
<td><%= @monster.max_level %></td>
<td><strong>Growth</strong></td>
<td><%= @monster.growth %></td>
</tr>
<tr>
<td><strong>Max Base HP</strong></td>
<td><%= @monster.maximum_base_hp %></td>
<td><strong>Speed</strong></td>
<td><%= @monster.speed %></td>
</tr>
<tr>
<td><strong>Max Base Strength</strong></td>
<td><%= @monster.maximum_base_strength %></td>
<td><strong>Max Base Magic</strong></td>
<td><%= @monster.maximum_base_magic %></td>
</tr>
<tr>
<td><strong>Immune to</strong></td>
<td><%= @monster.immune %></td>
<td><strong>Weak to</strong></td>
<td><%= @monster.weak %></td>
</tr>
<tr>
<td><strong>Halved for</strong></td>
<td><%= @monster.halved %></td>
<td><strong>Resistant to</strong></td>
<td><%= @monster.resistant %></td>
</tr>
<tr>
<td><strong>Feral Link</strong></td>
<td><%= @monster.feral_link %></td>
<td><strong>Description</strong></td>
<td><%= @monster.description %></td>
</tr>
<tr>
<td><strong>Effect</strong></td>
<td><%= @monster.effect %></td>
<td><strong>Damage Modifier</strong></td>
<td><%= @monster.damage_modifier %></td>
</tr>
<tr>
<td><strong>Charge Time</strong></td>
<td><%= @monster.charge_time.strftime("%H:%M") %></td>
<td/>
<td/>
</tr>
<tr>
<td><strong>PS3 Combo</strong></td>
<td><%= @monster.ps3_combo %></td>
<td><strong>Xbox 360 Combo</strong></td>
<td><%= @monster.xbox_360_combo %></td>
</tr>
<tr>
<td><strong>Notes</strong></td>
<td colspan="3"><%= @monster.special_notes %></td>
</tr></code></pre>
This is a lot of HTML, but it's not complicated, just tedious to enter. Notice that I kept the links and conditional statements for the locations so they would be clickable just like on the monster index page, and we have numerous additional monster attributes shown here. We do not yet have the list of abilities, but that list is going to be even more tedious to enter because there's almost 100 levels worth of abilities that need to be checked to conditionally include in this table. We'd much rather do that programmatically. Let's start by creating a sub-heading to set off this part of the table and create a row for the default passive abilities and skills:<br />
<pre><code class="html"> <tr class="table-dark">
<td/>
<td class="lead">Passive Abilities</td>
<td/>
<td class="lead">Skills</td>
</tr>
<tr>
<td><strong>Default</strong></td>
<td>
<% (1..4).each do |n| %>
<% if @monster.send "default_passive#{n}" %>
<%= @monster.send("default_passive#{n}").name %><br />
<% end %>
<% end %>
</td>
<td/>
<td>
<% (1..8).each do |n| %>
<% if @monster.send "default_skill#{n}" %>
<%= @monster.send("default_skill#{n}").name %><br />
<% end %>
<% end %>
</td>
</tr></code></pre>
Here we can see that the passive ability cell could list up to four abilities and the skill cell could list up to eight skills. Each ability is checked by creating the name of the method in the model for that ability as a string (e.g. <span style="font-family: "courier new" , "courier" , monospace;">"default_passive1"</span>), and sending that method name as a message. If the message returns a non-nil value, then we know it exists for that monster, and we can get its name to display in the table. That takes care of the default abilities for a monster, but the same task for the rest of the abilities is a bit more involved so we'll loop through each potential level, one per row, and call a helper method, a.k.a. a partial, to do the work:<br />
<pre><code class="html"> </tr>
<% (2..99).each do |n| %>
<%= render "level_ability", n: n, passive: "lv_%02d_passive" % [n], skill: "lv_%02d_skill" % [n] %>
<% end %>
</table></code></pre>
The <span style="font-family: "courier new" , "courier" , monospace;">render</span> method will look for a partial in <span style="font-family: "courier new" , "courier" , monospace;">app/views/monster/_level_ability.html.erb</span> and pass in three parameters: <span style="font-family: "courier new" , "courier" , monospace;">n</span>, <span style="font-family: "courier new" , "courier" , monospace;">passive</span>, and <span style="font-family: "courier new" , "courier" , monospace;">skill</span> with the corresponding values shown in the code. The level number needs to be a two-digit number, so I use a formatted string instead of string interpolation. Then, we can use these parameters in the partial, and it cleans up the code in the partial a bit:<br />
<pre><code class="html"> <% if @monster.has_ability?(passive) || @monster.has_ability?(skill) %>
<tr>
<td><strong>Level <%= "%02d" % [n] %></strong></td>
<td>
<% if @monster.has_ability? passive %>
<%= @monster.send(passive).name %><br />
<% end %>
</td>
<td/>
<td>
<% if @monster.has_ability? skill %>
<%= @monster.send(skill).name %><br />
<% end %>
</td>
</tr>
<% end %></code></pre>
Notice that in the case of each level ability, we first have to check if there are any passive abilities or skills at that level and only create a row if at least one of them exists. For any given monster, most levels will not have any abilities, and we don't want empty rows cluttering up the table. Then, we check that the ability exists again when creating the cell so that we don't get an exception for trying to call <span style="font-family: "courier new" , "courier" , monospace;">.name</span> on <span style="font-family: "courier new" , "courier" , monospace;">nil</span> for abilities that don't exist. So, why are we calling this <span style="font-family: "courier new" , "courier" , monospace;">has_ability?</span> method instead of trying to send the level ability method name to the model like we did before? Well, there are a few level abilities that don't exist for <i>any</i> monster, so trying to send a message with that method name will cause an exception. We have to be a bit more careful, so we can add this utility method to the Monster model in <span style="font-family: "courier new" , "courier" , monospace;">app/models/monster.rb</span> in order to do a safe check:<br />
<pre><code class="ruby"> def has_ability?(ability)
has_attribute?(ability + '_id') && send(ability)
end</code></pre>
The <span style="font-family: "courier new" , "courier" , monospace;">has_attribute?</span> check will make sure that that attribute exists before sending the message to the monster model to see if it's non-nil for that monster in particular. With that, we're ready to check on what our new monster details table looks like:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbBvjQcnDAQuH_M5QScEJvBosFi-sUfFP90ymYl_tANBlp8vIYf39hC7-WHZs_cE8BE-QP4dm4TKr85JKLdUnddS-7icRfAAo3RwdaiuIMUmxVE97P0vlRZNxZQHzexu9ipghtgcj8O8Y/s1600/monster_detail_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Monster detail page screenshot for Apkallu" border="0" data-original-height="692" data-original-width="1245" height="221" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbBvjQcnDAQuH_M5QScEJvBosFi-sUfFP90ymYl_tANBlp8vIYf39hC7-WHZs_cE8BE-QP4dm4TKr85JKLdUnddS-7icRfAAo3RwdaiuIMUmxVE97P0vlRZNxZQHzexu9ipghtgcj8O8Y/s400/monster_detail_screenshot.png" title="" width="400" /></a></div>
<br />
<h4>
Linking to Monster Details</h4>
<div>
That's a nice new monster detail page, but how do we get to it without typing in a random number for the monster in the address bar until we get what we want? It's as easy as adding a link to the monster names in the monster index table in <span style="font-family: "courier new" , "courier" , monospace;">app/views/monster/index.html.erb</span>:</div>
<pre><code class="html"> <% @monsters.each do |monster| %>
<tr>
<td><%= link_to monster.name, monster_path(monster) %></td>
<td><%= monster.role %></td></code></pre>
Rails will automatically insert the correct <span style="font-family: "courier new" , "courier" , monospace;">:id</span> into the URL based on which <span style="font-family: "courier new" , "courier" , monospace;">monster</span> object is passed to <span style="font-family: "courier new" , "courier" , monospace;">monster_path</span>. That's about the easiest thing we've done so far. Now we're ready to build out the ability tables.<br />
<br />
<h4>
Building the Ability Tables</h4>
<div>
Whew, more tables. We should be able to do this in our sleep right now, so I'll lay it out as quickly as I can. First, generate the controllers, views, and routes for the two models:</div>
<pre><code class="ruby">$ rails g controller Ability index
$ rails g controller RoleAbility index</code></pre>
Then, load the table data in each of the controllers in <span style="font-family: "courier new" , "courier" , monospace;">app/controllers/ability_controller.rb</span>:<br />
<pre><code class="ruby">class AbilityController < ApplicationController
def index
@abilities = Ability.all
end
end</code></pre>
And in <span style="font-family: "courier new" , "courier" , monospace;">app/controllers/role_ability_controller.rb</span>:<br />
<pre><code class="ruby">class RoleAbilityController < ApplicationController
def index
@role_abilities = RoleAbility.all
end
end</code></pre>
Also, build the table views in <span style="font-family: "courier new" , "courier" , monospace;">app/views/ability/index.html.erb</span>:<br />
<pre><code class="html"><h1>Monster Passive Abilities</h1>
<table id="ability-table" class="table table-striped table-sm">
<thead class="thead-dark">
<tr>
<th scope="col">Name</th>
<th scope="col">Rank</th>
<th scope="col">Description</th>
</tr>
</thead>
<% @abilities.each do |ability| %>
<tr>
<td><%= ability.name %></td>
<td><%= ability.rank %></td>
<td><%= ability.description %></td>
</tr>
<% end %>
</table></code></pre>
And in <span style="font-family: "courier new" , "courier" , monospace;">app/views/role_ability/index.html.erb</span>:<br />
<pre><code class="html"><h1>Monster Role Abilities</h1>
<table id="role-ability-table" class="table table-striped table-sm">
<thead class="thead-dark">
<tr>
<th scope="col">Name</th>
<th scope="col">Role</th>
<th scope="col">Infusable</th>
</tr>
</thead>
<% @role_abilities.each do |ability| %>
<tr>
<td><%= ability.name %></td>
<td><%= ability.role %></td>
<td><%= ability.infusable %></td>
</tr>
<% end %>
</table></code></pre>
<pre><code class="css"></code></pre>
We should also adjust the table width in <span style="font-family: "courier new" , "courier" , monospace;">app/assets/stylesheets/role_ability.scss</span>:<br />
<pre><code class="css">#role-ability-table {
width: 350px;
}</code></pre>
Finally, we add links in the home index in <span style="font-family: "courier new" , "courier" , monospace;">app/views/home/index.html.erb</span>:<br />
<pre><code class="html"><h1 class="display-3 text-center">Final Fantasy XIII-2</h1>
<h1 class="display-3 text-center">Monster Taming</h1>
<div class="container">
<div class="row">
<div class="col-sm"></div>
<div class="col-sm">
<ul class="list-group">
<%= link_to 'Monsters', monster_index_path,
class: "list-group-item list-group-item-action" %>
<%= link_to 'Monster Passive Abilities', ability_index_path,
class: "list-group-item list-group-item-action" %>
<%= link_to 'Monster Role Abilities', role_ability_index_path,
class: "list-group-item list-group-item-action" %>
<%= link_to 'Monster Materials', material_index_path,
class: "list-group-item list-group-item-action" %>
<%= link_to 'Monster Characteristics', characteristic_index_path,
class: "list-group-item list-group-item-action" %>
<%= link_to 'Game Locations', location_index_path,
class: "list-group-item list-group-item-action" %>
</ul>
</div>
<div class="col-sm"></div>
</div>
</div></code></pre>
And in the navigation bar in <span style="font-family: "courier new" , "courier" , monospace;">app/views/layouts/_header.html.erb</span>:<br />
<pre><code class="html"><nav class="navbar fixed-top navbar-expand navbar-dark bg-dark">
<%= link_to "FFXIII-2 Monster Taming", root_path, class: "navbar-brand" %>
<ul class="navbar-nav">
<li class="nav-item">
<%= link_to "Home", root_path, class: "nav-link" %>
</li>
<li class="nav-item">
<%= link_to "Monsters", monster_index_path, class: "nav-link" %>
</li>
<li class="nav-item">
<%= link_to "Abilities", ability_index_path, class: "nav-link" %>
</li>
<li class="nav-item">
<%= link_to "Role Abilities", role_ability_index_path, class: "nav-link" %>
</li>
<li class="nav-item">
<%= link_to "Materials", material_index_path, class: "nav-link" %>
</li>
<li class="nav-item">
<%= link_to "Characteristics", characteristic_index_path, class: "nav-link" %>
</li>
<li class="nav-item">
<%= link_to "Locations", location_index_path, class: "nav-link" %>
</li>
</ul>
</nav></code></pre>
Note that a few of the names were shortened to make space for the new links, and we have our last two tables built and ready to go:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7aFVpYc6c68Ik7SrKmTYzICr3_Ig4pCgn1CBjUwybQqHZTnsgMU1eaJaONFobGoCFVkGppsM4j_Vvy4iLChmvxoxZxdo6wszKj58MMg-dCTxOtTcY9SG0rnZdJMPZKxWgd3izf5bpDQ8/s1600/monster_passive_ability_table_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Monster passive ability table screenshot" border="0" data-original-height="679" data-original-width="1237" height="218" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7aFVpYc6c68Ik7SrKmTYzICr3_Ig4pCgn1CBjUwybQqHZTnsgMU1eaJaONFobGoCFVkGppsM4j_Vvy4iLChmvxoxZxdo6wszKj58MMg-dCTxOtTcY9SG0rnZdJMPZKxWgd3izf5bpDQ8/s400/monster_passive_ability_table_screenshot.png" title="" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNX9Kf5x8iPh3h79FudYR5XjgEe_EPZI2B9tkdHNitXKBleeWxCMGgb0LbEjxTtyJMDskOpYeIUBMbYd0LJTvjioqnkmwP9yeHbaLMtlB4pxcFDy1OhkLWfP98HM1TUNTvPDoBYxqZ764/s1600/monster_role_ability_table_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Monster role ability table screenshot" border="0" data-original-height="699" data-original-width="565" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNX9Kf5x8iPh3h79FudYR5XjgEe_EPZI2B9tkdHNitXKBleeWxCMGgb0LbEjxTtyJMDskOpYeIUBMbYd0LJTvjioqnkmwP9yeHbaLMtlB4pxcFDy1OhkLWfP98HM1TUNTvPDoBYxqZ764/s400/monster_role_ability_table_screenshot.png" title="" width="322" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h4>
Linking from Monsters to Abilities</h4>
<pre><code class="ruby"></code></pre>
The last thing we'll do before wrapping up for the day is to link up the monster abilities in the detailed monster pages to the ability tables we just created, similar to how we did it with the locations already. We'll use the same mechanism of passing the ability name as a parameter in the URL so that the corresponding row can be highlighted in the ability table. The links to the ability tables are added to the monster details view in both the default abilities in <span style="font-family: "courier new" , "courier" , monospace;">app/views/monster/show.html.erb</span>:<br />
<pre><code class="html"> <tr>
<td><strong>Default</strong></td>
<td>
<% (1..4).each do |n| %>
<% if @monster.send "default_passive#{n}" %>
<%= link_to @monster.send("default_passive#{n}").name,
ability_index_path(:highlight => @monster.send("default_passive#{n}").name) %><br />
<% end %>
<% end %>
</td>
<td/>
<td>
<% (1..8).each do |n| %>
<% if @monster.send "default_skill#{n}" %>
<%= link_to @monster.send("default_skill#{n}").name,
role_ability_index_path(:highlight => @monster.send("default_skill#{n}").name) %><br />
<% end %>
<% end %>
</td>
</tr>
<% (2..99).each do |n| %>
<%= render "level_ability", n: n, passive: "lv_%02d_passive" % [n], skill: "lv_%02d_skill" % [n] %>
<% end %>
</table></code></pre>
And each of the level abilities in <span style="font-family: "courier new" , "courier" , monospace;">app/views/monster/_level_ability.html.erb</span>:<br />
<pre><code class="html"> <% if @monster.has_ability?(passive) || @monster.has_ability?(skill) %>
<tr>
<td><strong>Level <%= "%02d" % [n] %></strong></td>
<td>
<% if @monster.has_ability? passive %>
<%= link_to @monster.send(passive).name,
ability_index_path(:highlight => @monster.send(passive).name) %><br />
<% end %>
</td>
<td/>
<td>
<% if @monster.has_ability? skill %>
<%= link_to @monster.send(skill).name,
role_ability_index_path(:highlight => @monster.send(skill).name) %><br />
<% end %>
</td>
</tr>
<% end %></code></pre>
This setup creates links from each passive ability to the passive ability table (obviously) and from each skill to the role ability table. Next, we need to make those :highlight parameters available to the views from the controllers in <span style="font-family: "courier new" , "courier" , monospace;">app/controllers/ability_controller.rb</span>:<br />
<pre><code class="ruby">class AbilityController < ApplicationController
def index
@abilities = Ability.all
@highlighted = params[:highlight]
end
end</code></pre>
And in <span style="font-family: "courier new" , "courier" , monospace;">app/controllers/role_ability_controller.rb</span>:<br />
<pre><code class="ruby">class RoleAbilityController < ApplicationController
def index
@role_abilities = RoleAbility.all
@highlighted = params[:highlight]
end
end</code></pre>
Finally, we can do the appropriate highlighting by adding a conditional class "<span style="font-family: "courier new" , "courier" , monospace;">table-primary</span>" to the corresponding row in the ability table views in <span style="font-family: "courier new" , "courier" , monospace;">app/views/ability/index.html.erb</span>:<br />
<pre><code class="html"> <% @abilities.each do |ability| %>
<tr <%= "class=table-primary" if ability.name == @highlighted %>>
<td><%= ability.name %></td>
<td><%= ability.rank %></td>
<td><%= ability.description %></td>
</tr>
<% end %>
</table></code></pre>
And in <span style="font-family: "courier new" , "courier" , monospace;">app/views/role_ability/index.html.erb</span>:<br />
<pre><code class="html"> <% @role_abilities.each do |ability| %>
<tr <%= "class=table-primary" if ability.name == @highlighted %>>
<td><%= ability.name %></td>
<td><%= ability.rank %></td>
<td><%= ability.description %></td>
</tr>
<% end %>
</table></code></pre>
Okay, wow. That was a lot of little steps to get all of that stuff built and hooked up, but it wasn't too difficult. Now we have a great set of tables that we can make improvements to by interconnecting them more, and we can start building in features to make answering the more complex questions about monster taming a bit easier to do.<br />
<br />
Before we get to those advanced features, we'll want to fix a couple things with our new ability tables, though. For one, the passive ability table in particular is too big to just highlight the row for the linked ability and require the user to go scrolling through to find it. We'll make that feature more user-friendly next time. Additionally, we don't have a link back to monsters from the ability tables. Wouldn't it be nice to click on an ability and have the website show a list of monsters with that ability? It's not as simple as it was with filtering the monster table by location because the abilities are spread across hundreds of attributes, but we'll figure that out next time as well.<div><br /></div><div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div></div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-49947237477557126842020-06-29T20:45:00.001-05:002020-10-26T20:19:35.918-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2: Viewing the Monster DataLast time in this Final Fantasy XIII-2 monster taming mechanics series, we continued building views and controllers in the Rails MVC architecture for <a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html" target="_blank">the monster characteristics and game locations tables</a>. Now it's time to tackle the main table of the site: the monster table. Once we have this monster table, we'll want to add links to the elements of the table so that we can jump directly to related tables of interest. We'll learn how to do that task as well, and we'll see just how easy Rails makes it.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjV7g-L19_ftvDAmDXXwVXd4Q6Q7FgxZ_6qYMtvWL8y5j5w_j1nuclUC11_JCOx-9sQrl56H0-CoSvb8wRxApfOajtHUyKOwr8SM-_vbnOKf58G9TPOgExgi1H91GeVNLT11WF7aKg7s6k/s1600/ffxiii-2_battle_scene.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Final Fantasy XIII-2 battle scene" border="0" data-original-height="720" data-original-width="1280" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjV7g-L19_ftvDAmDXXwVXd4Q6Q7FgxZ_6qYMtvWL8y5j5w_j1nuclUC11_JCOx-9sQrl56H0-CoSvb8wRxApfOajtHUyKOwr8SM-_vbnOKf58G9TPOgExgi1H91GeVNLT11WF7aKg7s6k/s400/ffxiii-2_battle_scene.jpg" title="" width="400" /></a></div>
<br />
<a name='more'></a><h4>
Create a Monster Page</h4>
<script>hljs.initHighlightingOnLoad();</script>
Creating a stand-alone monster page is as simple as it was to <a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html" target="_blank">create the other pages</a>. Let's do that first so we'll have something to work with. This will be a rapid set of steps for this now-routine process. First, we run the rails controller generator for it:<br />
<pre><code class="csh">$ rails g controller Monster index</code></pre>
Remember, this creates <span style="font-family: "courier new" , "courier" , monospace;">Monster</span> controller and view files and adds the route for the view page to the top of the <span style="font-family: "courier new" , "courier" , monospace;">config/routes.rb</span> file:<br />
<pre><code class="ruby">Rails.application.routes.draw do
get 'monster/index'
get 'location/index'
get 'characteristic/index'
get 'material/index'
get 'home/index'
root 'home#index'
end</code></pre>
Next, load the table data into an instance variable in the controller in <span style="font-family: "courier new" , "courier" , monospace;">app/controllers/monster_controller.rb</span>:<br />
<pre><code class="ruby">class MonsterController < ApplicationController
def index
@monsters = Monster.all
end
end</code></pre>
Then, duplicate one of the other table's views we created last time in <span style="font-family: "courier new" , "courier" , monospace;">app/views/monster/index.html.erb</span> and make the necessary modifications for the monster table:<br />
<pre><code class="html"><h1>Monsters</h1>
<table id="monster-table" class="table table-striped table-sm">
<thead class="thead-dark">
<tr>
<th scope="col">Name</th>
<th scope="col">Role</th>
<th scope="col">Location</th>
<th scope="col">Tame Rate</th>
<th scope="col">Max Level</th>
<th scope="col">Base HP</th>
<th scope="col">Base Strength</th>
<th scope="col">Base Magic</th>
<th scope="col">Speed</th>
<th scope="col">Growth</th>
<th scope="col">Immune</th>
<th scope="col">Resistant</th>
<th scope="col">Halved</th>
<th scope="col">Weak</th>
</tr>
</thead>
<% @monsters.each do |monster| %>
<tr>
<td><%= monster.name %></td>
<td><%= monster.role %></td>
<td><%= monster.location.name %></td>
<td><%= monster.tame_rate %></td>
<td><%= monster.max_level %></td>
<td><%= monster.max_base_hp %></td>
<td><%= monster.max_base_strength %></td>
<td><%= monster.max_base_magic %></td>
<td><%= monster.speed %></td>
<td><%= monster.growth %></td>
<td><%= monster.immune %></td>
<td><%= monster.resistant %></td>
<td><%= monster.halved %></td>
<td><%= monster.weak %></td>
</tr>
<% end %>
</table>
</code></pre>
Notice we didn't include all of the properties for each monster because it would be far too many cells for this table. Remember, there are 100 levels for skill and passive abilities, each! It would be information overload, so we're going to have to come up with another way to view the details for each monster, and just show some of the most important properties in this table. (We'll get to that task next time.) Second, the <span style="font-family: "courier new" , "courier" , monospace;">monster.location</span> is not a string, like the other attributes. It's actually a model object for the location, and we can pull up its name by going one level deeper with <span style="font-family: "courier new" , "courier" , monospace;">monster.location.name</span>. This mechanism is enabled by the <span style="font-family: "courier new" , "courier" , monospace;">belongs_to</span> reference that was set up in the <a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html" target="_blank">Monster model several articles back</a>. Finally, we don't need to add extra formatting to the CSS file at <span style="font-family: "courier new" , "courier" , monospace;">app/assets/stylesheets/monsters.scss</span> to tighten up the table because the table needs to be plenty wide for all of these columns.<br />
<br />
We should also add this page as another link to our index in <span style="font-family: "courier new" , "courier" , monospace;">app/views/home/index.html.erb</span> so we can more easily get to it:<br />
<pre><code class="html"><h1 class="display-3 text-center">Final Fantasy XIII-2</h1>
<h1 class="display-3 text-center">Monster Taming</h1>
<div class="container">
<div class="row">
<div class="col-sm"></div>
<div class="col-sm">
<ul class="list-group">
<%= link_to 'Monsters', monster_index_path,
class: "list-group-item list-group-item-action" %>
<%= link_to 'Monster Materials', material_index_path,
class: "list-group-item list-group-item-action" %>
<%= link_to 'Monster Characteristics', characteristic_index_path,
class: "list-group-item list-group-item-action" %>
<%= link_to 'Game Locations', location_index_path,
class: "list-group-item list-group-item-action" %>
</ul>
</div>
<div class="col-sm"></div>
</div>
</div></code></pre>
And let's not forget the new navigation bar. The monster page is missing from the navbar in <span style="font-family: "courier new" , "courier" , monospace;">app/views/layouts/_header.html.erb</span> as well:<br />
<pre><code class="html"><nav class="navbar fixed-top navbar-expand navbar-dark bg-dark">
<%= link_to "FFXIII-2 Monster Taming", root_path,
class: "navbar-brand" %>
<ul class="navbar-nav">
<li class="nav-item">
<%= link_to "Home", root_path, class: "nav-link" %>
</li>
<li class="nav-item">
<%= link_to "Monsters", monster_index_path,
class: "nav-link" %>
</li>
<li class="nav-item">
<%= link_to "Monster Materials", material_index_path,
class: "nav-link" %>
</li>
<li class="nav-item">
<%= link_to "Monster Characteristics", characteristic_index_path,
class: "nav-link" %>
</li>
<li class="nav-item">
<%= link_to "Game Locations", location_index_path,
class: "nav-link" %>
</li>
</ul>
</nav></code></pre>
And finally, we have another pretty table:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbmS7B4LFssjHHw_FOpHbe_xR5deEFM6N8TVZmPpi5fGImbqZrjE-9IVAtGjjUAhp0R4r7vR_yga42tEdEtH1jar9nlCA06ziQd8BjnTYFejvEgMQe54ni2QEGz8k24NvNYyF_wOd4peU/s1600/monster_table.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of monster table" border="0" data-original-height="612" data-original-width="1289" height="188" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbmS7B4LFssjHHw_FOpHbe_xR5deEFM6N8TVZmPpi5fGImbqZrjE-9IVAtGjjUAhp0R4r7vR_yga42tEdEtH1jar9nlCA06ziQd8BjnTYFejvEgMQe54ni2QEGz8k24NvNYyF_wOd4peU/s400/monster_table.png" title="" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<h4>
Linking Tables Together</h4>
<div>
With the introduction of this monster table, we are now able to start linking these tables together. We see right away that monsters are in a location, so we can link the cells in the location column to the game location table. To set up that link, we can start by simply adding a link to the monster table view around the text in the location cells:<br />
<pre><code class="html"> <% @monsters.each do |monster| %>
<tr>
<td><%= monster.name %></td>
<td><%= monster.role %></td>
<td><%= link_to monster.location.name, location_index_path %></td>
<td><%= monster.tame_rate %></td>
<td><%= monster.max_level %></td>
<td><%= monster.max_base_hp %></td>
<td><%= monster.max_base_strength %></td>
<td><%= monster.max_base_magic %></td>
<td><%= monster.speed %></td>
<td><%= monster.growth %></td>
<td><%= monster.immune %></td>
<td><%= monster.resistant %></td>
<td><%= monster.halved %></td>
<td><%= monster.weak %></td>
</tr>
<% end %>
</table></code></pre>
And as easy as that, we have links from each location to the location table:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3awmrH1GPKTRpsPneppQLK5O9q4phDQ4d9VDPz_dkclJzBQ0wJiZEOz_55NzyVSXOONzOwYn0Gzek7Gqs4s-1QpdB0t55Efu3xUZSdpO8rSHsJcRHy-9yGOzgwCqoNoRrDZ_ob5eW0XE/s1600/monster_table_location_links_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of monster table with location links" border="0" data-original-height="524" data-original-width="1117" height="187" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3awmrH1GPKTRpsPneppQLK5O9q4phDQ4d9VDPz_dkclJzBQ0wJiZEOz_55NzyVSXOONzOwYn0Gzek7Gqs4s-1QpdB0t55Efu3xUZSdpO8rSHsJcRHy-9yGOzgwCqoNoRrDZ_ob5eW0XE/s400/monster_table_location_links_screenshot.png" title="" width="400" /></a></div>
<br />
This addition was the bare minimum, though, since all links go to the same table, and then we have to search in the location table for the game area that we actually want to look at. Since it wouldn't make sense to have the link point to a different page for each location, as the location only has one other property, its source area, we should do something else to make the location clicked on easier to find in the table. Maybe we could highlight it? That's also pretty straightforward to do. First, we need to add a parameter to the link so that the location page knows which location to highlight:<br />
<pre><code class="html"> <% @monsters.each do |monster| %>
<tr>
<td><%= monster.name %></td>
<td><%= monster.role %></td>
<td>
<%= link_to monster.location.name,
location_index_path(:highlight => monster.location.name) %>
</td>
<td><%= monster.tame_rate %></td>
<td><%= monster.max_level %></td>
<td><%= monster.max_base_hp %></td>
<td><%= monster.max_base_strength %></td>
<td><%= monster.max_base_magic %></td>
<td><%= monster.speed %></td>
<td><%= monster.growth %></td>
<td><%= monster.immune %></td>
<td><%= monster.resistant %></td>
<td><%= monster.halved %></td>
<td><%= monster.weak %></td>
</tr>
<% end %>
</table&gt</code></pre>
Then, we need to make that parameter available in the location controller index action to the view:<br />
<pre><code class="ruby">class LocationController < ApplicationController
def index
@location = Location.all
@highlighted = params[:highlight]
end
end</code></pre>
The <span style="font-family: "courier new" , "courier" , monospace;">params</span> hash table contains any parameters present in the URL for the page, so we can extract the <span style="font-family: "courier new" , "courier" , monospace;">:highlight</span> parameter and put it in the instance variable <span style="font-family: "courier new" , "courier" , monospace;">@highlighted</span>. It'll be nil if it's not present in the URL. Finally, we use that variable to highlight the correct row with a Bootstrap class in the location view:<br />
<pre><code class="html"> <% @locations.each do |location| %>
<tr <%= "class=table-primary" if location.name == @highlighted %>>
<td><%= location.name %></td>
<td><%= location.source&.name %></td>
</tr>
<% end %>
</table></code></pre>
The <span style="font-family: "courier new" , "courier" , monospace;">table-primary</span> class is only added to the row that has the location name corresponding to what's in <span style="font-family: "courier new" , "courier" , monospace;">@highlighted</span>. If <span style="font-family: "courier new" , "courier" , monospace;">@highlighted</span> is nil, then obviously nothing gets highlighted. Now when we click on a location link, the corresponding row is indeed highlighted in the location table:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTjABq1pq4jp2ijUi0D3SL70yFih-aRUaIxf-zbNEIuo14jCk8d2CaGIie9LmVAl2NXoYBj6pirq4koOqhYwwELK-qJs24N9QB7vnToJwVO2i1mz-l863fJYX4mPjtxTn9czL3uEvLEjg/s1600/location_table_highlighted_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of highlighted row in location table" border="0" data-original-height="465" data-original-width="513" height="362" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTjABq1pq4jp2ijUi0D3SL70yFih-aRUaIxf-zbNEIuo14jCk8d2CaGIie9LmVAl2NXoYBj6pirq4koOqhYwwELK-qJs24N9QB7vnToJwVO2i1mz-l863fJYX4mPjtxTn9czL3uEvLEjg/s400/location_table_highlighted_screenshot.png" title="" width="400" /></a></div>
<br />
Nice, but we're not done. Looking back at the monster table, notice that we only included the first location for each monster, but there are potentially up to three locations a monster can be in. We want to add those other locations to the table when they exist. To add links for <span style="font-family: "courier new" , "courier" , monospace;">location2</span> and <span style="font-family: "courier new" , "courier" , monospace;">location3</span> safely, we need to conditionally add them only if they're not nil, and this is how we do that in the monster view template:<br />
<pre><code class="html"> <% @monsters.each do |monster| %>
<tr>
<td><%= monster.name %></td>
<td><%= monster.role %></td>
<td>
<%= link_to monster.location.name,
location_index_path(:highlight => monster.location.name) %>
<% if monster.location2 %>
<%= link_to monster.location2.name,
location_index_path(:highlight => monster.location2.name) %>
<% end %>
<% if monster.location3 %>
<%= link_to monster.location3.name,
location_index_path(:highlight => monster.location3.name) %>
<% end %>
</td>
<td><%= monster.tame_rate %></td>
<td><%= monster.max_level %></td>
<td><%= monster.max_base_hp %></td>
<td><%= monster.max_base_strength %></td>
<td><%= monster.max_base_magic %></td>
<td><%= monster.speed %></td>
<td><%= monster.growth %></td>
<td><%= monster.immune %></td>
<td><%= monster.resistant %></td>
<td><%= monster.halved %></td>
<td><%= monster.weak %></td>
</tr>
<% end %>
</table>
</code></pre>
We simply need to wrap each link in a condition with <span style="font-family: "courier new" , "courier" , monospace;"><%...%></span> template tags so the if-condition will run as Ruby code without returning any output to the view HTML. If the condition is false, meaning the location object is nil, then it will skip over creating the link to the following <span style="font-family: "courier new" , "courier" , monospace;"><% end %></span>. So, the template renderer actually understands Ruby code that crosses template tag boundaries. If we wanted to write multiple lines of Ruby code that either all generated HTML output or all didn't generate output, we could span lines with a single template tag, but here we needed to alternate between output and no output, so each tag is one Ruby statement. We can take advantage of the fact that the renderer crosses the template tag boundaries to make it all work. If we now take a look at our table, we see further down the list that we do indeed have two and three locations in some cells:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgX73hdY9T0JHtIWlBUlVA8W5IWNDfDWyHD4XF_HSCnj58mWLpRCYrTa3th592fmuOd14PRWcM9M2GoAH10aRZqDmdXmZUpqQHInBYqaIUxKfZ2lLXnLXN7KJUcPHmR3DaI_0LAvPx9Vik/s1600/monster_table_multi-location_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of monster table with multi-location cells" border="0" data-original-height="452" data-original-width="1131" height="158" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgX73hdY9T0JHtIWlBUlVA8W5IWNDfDWyHD4XF_HSCnj58mWLpRCYrTa3th592fmuOd14PRWcM9M2GoAH10aRZqDmdXmZUpqQHInBYqaIUxKfZ2lLXnLXN7KJUcPHmR3DaI_0LAvPx9Vik/s400/monster_table_multi-location_screenshot.png" title="" width="400" /></a></div>
<br />
Each link can be clicked to bring us to the location table with the appropriate row highlighted according to the source link. Just like we wanted, nice!<br />
<br />
<h4>
Linking Back to Monsters</h4>
We could fiddle with many more features in the monster table, but we've finished linking the monster table to the location table. It's time we think about linking back from the location table to the monster table. Remember that the monster table is much longer than the location table, and numerous monsters are going to be in any given location, so we don't want to just highlight all of the monster rows from the location that we clicked a link on. That would be tedious to scroll around the monster table finding all of the highlighted monsters strewn across the table. Instead, let's filter the table and show that filtered view. First, we can add links to the location view with the filter parameter included:<br />
<pre><code class="html"> <% @locations.each do |location| %>
<tr <%= "class=table-primary" if location.name == @highlighted %>>
<td>
<%= link_to location.name,
monster_index_path(:filter => location.name) %>
</td>
<td><%= location.source&.name %></td>
</tr>
<% end %>
</table></code></pre>
We send the location's name as the filter again here because we just want to send one name in the link, not all of the names for the monsters that can be found in that area. We'll figure out which monsters to show in the monster controller, like so:<br />
<pre><code class="ruby">class MonsterController < ApplicationController
def index
if params[:filter]
location = Location.find_by(name: params[:filter])
@monsters = location.location_monsters +
location.location2_monsters +
location.location3_monsters
else
@monsters = Monster.all
end
end
end</code></pre>
First, we check if there's a <span style="font-family: "courier new" , "courier" , monospace;">:filter</span> parameter in the URL. If there is, we grab the object for that location name. (It's currently always a location name; maybe later we'll have to be more specific with our filters if we add more.) Then, we concatenate all of the monsters that can be found in that location, making sure to include the secondary and tertiary locations. We can use these associations because we set them up already in the location model with the <span style="font-family: "courier new" , "courier" , monospace;">has_many</span> designation:<br />
<pre><code class="ruby">class Location < ApplicationRecord
belongs_to :source, class_name: 'Location', optional: true
has_many :location_monsters, :class_name => 'Monster', :foreign_key => 'location'
has_many :location2_monsters, :class_name => 'Monster', :foreign_key => 'location2'
has_many :location3_monsters, :class_name => 'Monster', :foreign_key => 'location3'
end</code></pre>
Now we can check to see if the filter works by going to the location page and clicking on a location, say "Academia 500 AF," and we get a smaller table of just the monsters found in that area:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCHMmdZjSLq_7BOc_bztO6P8f_iglvB6ld7Iif7AtviZV4nRanQw4-J6VZtfY8qBpSUs82QIUiJfqkvINaGQApbdOtcjA6L7wW0FDRhCqqfkjt47vrjPCEH_SJBxbsBt-9dknxmjyawL8/s1600/monster_table_filtered_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of monster table filtered by a location" border="0" data-original-height="581" data-original-width="1126" height="206" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCHMmdZjSLq_7BOc_bztO6P8f_iglvB6ld7Iif7AtviZV4nRanQw4-J6VZtfY8qBpSUs82QIUiJfqkvINaGQApbdOtcjA6L7wW0FDRhCqqfkjt47vrjPCEH_SJBxbsBt-9dknxmjyawL8/s400/monster_table_filtered_screenshot.png" title="" width="400" /></a></div>
<br />
That's pretty slick, and we only had to change a few lines of code to accomplish it. Rails is just so enjoyable to work with, and things are starting to get interesting. We're beginning to be able to answer some questions about the game just by browsing through the tables, like "which monsters can I find in this area?" And "how do I get to this location where such-and-such a monster is?" That's real progress! Since we didn't get to displaying all of the monster attributes, we'll do that next time, as well as starting in on our last two tables - the ability tables.</div><div><br /></div><div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div></div>
Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-37489599811487548862020-06-16T20:38:00.001-05:002020-10-26T20:19:21.806-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2: Viewing More DataIn the previous episode of this Final Fantasy XIII-2 monster taming mechanics series, we started looking at the view and controller parts of the Rails MVC architecture by <a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html" target="_blank">building views of the monster material table and the site index</a>. Now it's time to expand our views to some more of the simpler tables, and in so doing, we'll need to improve our site navigation so that we don't have to keep going back to the home page to get anywhere else. We'll build up the monster characteristics and game locations views so that we have something to fill out our navigation, and then we can see how easy it is to build a site-wide navigation bar using Bootstrap. Let's get started.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhoFTN_YUZr-4SoGTQdhcymVngoSi1oYFmgqfUVFd6rEUfT_x4nlo75iGRJ58_G_0Y0A7U5mXAHmkelCnZRRbQg-B3anSepBopIaqHdqMLGvVRzq99vgTPxWygXdi6K8bF_cd3dWs-uZgI/s1600/XIII2battle_cactuar_gold_chocobo.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Final Fantasy XIII-2 battle scene with Cactuar and Chocobo" border="0" data-original-height="343" data-original-width="610" height="224" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhoFTN_YUZr-4SoGTQdhcymVngoSi1oYFmgqfUVFd6rEUfT_x4nlo75iGRJ58_G_0Y0A7U5mXAHmkelCnZRRbQg-B3anSepBopIaqHdqMLGvVRzq99vgTPxWygXdi6K8bF_cd3dWs-uZgI/s400/XIII2battle_cactuar_gold_chocobo.jpg" title="" width="400" /></a></div>
<br />
<a name='more'></a><h4>
Create a Monster Characteristics Page</h4>
<script>hljs.initHighlightingOnLoad();</script>
Creating the monster characteristics page is as simple as it was to create the monster materials page last time. First, we run the rails controller generator for it:<br />
<pre><code class="csh">$ rails g controller Characteristic index</code></pre>
Remember, this creates <span style="font-family: "courier new" , "courier" , monospace;">Characteristic</span> controller and view files and adds the route to the view page to the top of the <span style="font-family: "courier new" , "courier" , monospace;">config/routes.rb</span> file:<br />
<pre><code class="ruby">Rails.application.routes.draw do
get 'characteristic/index'
get 'material/index'
get 'home/index'
root 'home#index'
end</code></pre>
With this addition to the routes, we can load the page at <span style="font-family: "courier new" , "courier" , monospace;">http://localhost:3000/characteristic/index</span> after starting the Rails server, but first we want to load the table data in the controller and render it in a table in the view. First, the controller in <span style="font-family: "courier new" , "courier" , monospace;">app/controllers/characteristic_controller.rb</span>:<br />
<pre><code class="ruby">class CharacteristicController < ApplicationController
def index
@characteristics = Characteristic.all
end
end</code></pre>
With all the characteristics loaded into the <span style="font-family: "courier new" , "courier" , monospace;">@characteristics</span> instance variable, we can make a nice HTML table by duplicating the material view table we created last time in <span style="font-family: "courier new" , "courier" , monospace;">app/views/characteristic/index.html.erb</span> and making the necessary modifications:<br />
<pre><code class="html"><h1>Monster Characteristics</h1>
<table id="characteristic-table" class="table table-striped table-sm">
<thead class="thead-dark">
<tr>
<th scope="col">Name</th>
<th scope="col">Description</th>
</tr>
</thead>
<% @characteristics.each do |characteristic| %>
<tr>
<td><%= characteristic.name %></td>
<td><%= characteristic.description %></td>
</tr>
<% end %>
</table>
</code></pre>
Finally, we add the same formatting to the CSS file at app/assets/stylesheets/characteristic.scss to tighten up the table a bit:<br />
<pre><code class="css">#characteristic-table {
width: 700px;
}</code></pre>
We've quickly built another pretty table:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL3rqF68TavopwcD20JVPFwXc1IS9huKHan-ZIgIir-k_eSPbHM9jh54KU9oKsRXJ08AAdQnADqZUCBZZu-Kv8uqSIWfxZ9iOQOlBL1vNYs5LsHT_1vlymJ7HtsfL08KBjNos5xW2ki_0/s1600/monster_characteristics_table.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Monster Characteristics table" border="0" data-original-height="660" data-original-width="710" height="371" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL3rqF68TavopwcD20JVPFwXc1IS9huKHan-ZIgIir-k_eSPbHM9jh54KU9oKsRXJ08AAdQnADqZUCBZZu-Kv8uqSIWfxZ9iOQOlBL1vNYs5LsHT_1vlymJ7HtsfL08KBjNos5xW2ki_0/s400/monster_characteristics_table.png" title="" width="400" /></a></div>
<br />
We should also add this page as another link to our index in <span style="font-family: "courier new" , "courier" , monospace;">app/views/home/index.html.erb</span> so we can more easily get to it:<br />
<pre><code class="html"><h1>Final Fantasy XIII-2 Monster Taming</h1>
<%= link_to 'Monster Materials', material_index_path %>
<%= link_to 'Monster Characteristics', characteristic_index_path %></code></pre>
<br />
<h4>
Create a Game Location Page</h4>
Just to weigh down our site navigation a bit more and show that we really need something better than the back button, let's add in one more table for the game locations. First, we'll run the generator again, and then I'll quickly list off the file changes since this should all be routine now:<br />
<pre><code class="csh">$ rails g controller Location index</code></pre>
This command creates a Location controller and view, and adds a route to the top of the <span style="font-family: "courier new" , "courier" , monospace;">config/routes.rb</span> file:<br />
<pre><code class="ruby">Rails.application.routes.draw do
get 'location/index'
get 'characteristic/index'
get 'material/index'
get 'home/index'
root 'home#index'
end</code></pre>
Next, load the location data in <span style="font-family: "courier new" , "courier" , monospace;">app/controllers/location_controller.rb</span>:<br />
<pre><code class="ruby">class LocationController < ApplicationController
def index
@locations = Location.all
end
end</code></pre>
Then, make the HTML table in <span style="font-family: "courier new" , "courier" , monospace;">app/views/location/index.html.erb</span>:<br />
<pre><code class="html"><h1>Game Locations</h1>
<table id="location-table" class="table table-striped table-sm">
<thead class="thead-dark">
<tr>
<th scope="col">Name</th>
<th scope="col">Source Area</th>
</tr>
</thead>
<% @locations.each do |location| %>
<tr>
<td><%= location.name %></td>
<td><%= location.source&.name %></td>
</tr>
<% end %>
</table>
</code></pre>
Notice the special '&.' operator in <span style="font-family: "courier new" , "courier" , monospace;">location.source&.name</span>? This operator was used because <span style="font-family: "courier new" , "courier" , monospace;">location.source</span> is another location object so we need to get that location's name, but in the case of New Bodhum 003 AF, there is no source location. The '&.' operator will stop and return this nil value instead of trying to call <span style="font-family: "courier new" , "courier" , monospace;">.name</span> on nil, which would be bad. Thus, we have a nice clean syntax instead of needing to write <span style="font-family: "courier new" , "courier" , monospace;">location.source && location.source.name</span>. Moving on, we finally add the table formatting to <span style="font-family: "courier new" , "courier" , monospace;">app/assets/stylesheets/location.scss</span>:<br />
<pre><code class="css">#location-table {
width: 500px;
}</code></pre>
We've quickly built yet another pretty table:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQ-3k_1HWYXCfmL0zi33tuTVWWE9ffntCfkIxRFcPum6O1vOttYy8iyT9xmSs13TvxKTcQF3yk_A8-zqEs0l_Z-UyA09iaUqGeEceoR48ov5E-_q4y019dlmuIgYgMhWjwbRe2itXhOvU/s1600/game_locations_table.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Game Locations table" border="0" data-original-height="529" data-original-width="509" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQ-3k_1HWYXCfmL0zi33tuTVWWE9ffntCfkIxRFcPum6O1vOttYy8iyT9xmSs13TvxKTcQF3yk_A8-zqEs0l_Z-UyA09iaUqGeEceoR48ov5E-_q4y019dlmuIgYgMhWjwbRe2itXhOvU/s400/game_locations_table.png" title="" width="383" /></a></div>
<br />
Eventually, we're going to want to link this table and the monster characteristics table back and forth with the main monster table data so that we can list all of the monsters in a selected location or with a selected characteristic or find all the locations where a selected monster can be found, but for now these are just stand-alone tables.<br />
<br />
<h4>
Rethinking Navigation</h4>
After adding the game location page to the index at <span style="font-family: "courier new" , "courier" , monospace;">app/views/home/index.html.erb</span>:<br />
<pre><code class="html"><h1>Final Fantasy XIII-2 Monster Taming</h1>
<%= link_to 'Monster Materials', material_index_path %>
<%= link_to 'Monster Characteristics', characteristic_index_path %>
<%= link_to 'Game Locations', location_index_path %></code></pre>
I take a look at my index page and realize it's very plain:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiX98vP55uH31Zdqc1bvQ-Z_uVzRsNMsRKv_kLuFxfM2_jOmtMtGp73_J5CQ8wZKiJlOjP6CAd80G2U5RemXpQIwuuroO023YODKxdR2zbX-ndAJSw5p1eOJerzrA1dCUnHtemVIpR9uPU/s1600/plain_index_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Plain index screenshot" border="0" data-original-height="129" data-original-width="654" height="78" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiX98vP55uH31Zdqc1bvQ-Z_uVzRsNMsRKv_kLuFxfM2_jOmtMtGp73_J5CQ8wZKiJlOjP6CAd80G2U5RemXpQIwuuroO023YODKxdR2zbX-ndAJSw5p1eOJerzrA1dCUnHtemVIpR9uPU/s400/plain_index_screenshot.png" title="" width="400" /></a></div>
<br />
We can certainly do better, both visually and functionally. Visually, we'll space things out more to make it more readable. Functionally, we want to be able to hit those links more easily so we're going to make them bigger and pull them off the edge of the page. Beyond the index page, our navigation is non-existent. Every time we select a table to look at, the only ways to get to another table are to click the back button in the browser or (shudder) type the correct address into the address bar. We're going to fix that by adding a global navigation bar to all of the pages.<br />
<br />
First, let's tackle that index page. Looking over Bootstrap's page component options, I kind of like the list group links for the list of tables and the display heading for a nice, big title for the page. We can change the index to use these Bootstrap components easily enough by adding the appropriate HTML elements and classes:<br />
<pre><code class="html"><h1 class="display-3 text-center">Final Fantasy XIII-2</h1>
<h1 class="display-3 text-center">Monster Taming</h1>
<div class="container">
<div class="row">
<div class="col-sm"></div>
<div class="col-sm">
<ul class="list-group">
<%= link_to 'Monster Materials', material_index_path,
class: "list-group-item list-group-item-action" %>
<%= link_to 'Monster Characteristics', characteristic_index_path,
class: "list-group-item list-group-item-action" %>
<%= link_to 'Game Locations', location_index_path,
class: "list-group-item list-group-item-action" %>
</ul>
</div>
<div class="col-sm"></div>
</div>
</div></code></pre>
This may look like a lot more structure, but it's there to make use of Bootstrap's grid layout. The extra <span style="font-family: "courier new" , "courier" , monospace;"><div></span> elements set up a grid with three columns for the list of links, and the links are put in the center column so that the border around the links themselves doesn't stretch all the way across the page. The necessary classes for the links are added to the <span style="font-family: "courier new" , "courier" , monospace;">link_to</span> methods with the class option, making it easy to add the desired formatting to links. Now our index page has some visual punch and is easier to use as well:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqKlSZU2Ta4LeOq03sGCEYDbKeYL_x4dCoGr03aSep_RCnYypmhaEHLoA6qcm1AEMNKntcdVQH4rCw5pVoUfYXMCdjNG7-2e5RZWU3UlBvby1tAVHXETyCZrBUtprqfRampNfInSbV110/s1600/fancy_index_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Fancy index screenshot" border="0" data-original-height="411" data-original-width="666" height="245" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqKlSZU2Ta4LeOq03sGCEYDbKeYL_x4dCoGr03aSep_RCnYypmhaEHLoA6qcm1AEMNKntcdVQH4rCw5pVoUfYXMCdjNG7-2e5RZWU3UlBvby1tAVHXETyCZrBUtprqfRampNfInSbV110/s400/fancy_index_screenshot.png" title="" width="400" /></a></div>
<br />
Okay, I'm sure there's plenty more we could do with this to make it more pleasing to the eye, but I'm an engineer, not a designer. We can move on to the global navigation. For this component, we're going to want to use a fixed navbar with a title element and a set of links for the different table pages. We'll start with a basic navbar for now, and we may need to extend it with some drop-down menus or a search form after we add more features to the site. Since we want this navbar to appear on every page, but we don't want to repeat the HTML everywhere, we should put it somewhere that will be rendered for every page auto-magically. Luckily, Rails has just such a place in <span style="font-family: "courier new" , "courier" , monospace;">app/views/layouts/application.html.erb</span>:<br />
<pre><code class="html"> <body>
<%= render 'layouts/header' %>
<main class="container">
<%= yield %>
</main>
</body></code></pre>
The HTML inside the <span style="font-family: "courier new" , "courier" , monospace;"><body></span> tag used to just be the <span style="font-family: "courier new" , "courier" , monospace;"><%= yield %></span> statement that runs the appropriate controller for the page that's loading and renders the view for that page from the appropriate view file. We don't want to just drop all of the navbar code in this file, so we add a helper file and tell the renderer to fetch it before rendering the rest of the page. We also wrap a <span style="font-family: "courier new" , "courier" , monospace;"><main></span> tag with a Bootstrap class of <span style="font-family: "courier new" , "courier" , monospace;">container</span> around the rest of the page so that we can more easily control the global formatting of the pages. Adding this container class adds some better page margin spacing by default.<br />
<br />
Getting back to that header file, it actually resides at <span style="font-family: "courier new" , "courier" , monospace;">app/views/layouts/_header.html.erb</span> because all helper views get an underscore prepended to them to differentiate them from regular views. That header with our navbar looks like this:<br />
<pre><code class="html"><nav class="navbar fixed-top navbar-expand navbar-dark bg-dark">
<%= link_to "FFXIII-2 Monster Taming", root_path,
class: "navbar-brand" %>
<ul class="navbar-nav">
<li class="nav-item">
<%= link_to "Home", root_path, class: "nav-link" %>
</li>
<li class="nav-item">
<%= link_to "Monster Materials", material_index_path,
class: "nav-link" %>
</li>
<li class="nav-item">
<%= link_to "Monster Characteristics", characteristic_index_path,
class: "nav-link" %>
</li>
<li class="nav-item">
<%= link_to "Game Locations", location_index_path,
class: "nav-link" %>
</li>
</ul>
</nav></code></pre>
Okay, what does all of this stuff do? Starting with the <span style="font-family: "courier new" , "courier" , monospace;"><nav></span> tag, the classes work as follows:<br />
<br />
<ul>
<li><span style="font-family: "courier new" , "courier" , monospace;">navbar</span>: applies general navigation bar formatting</li>
<li><span style="font-family: "courier new" , "courier" , monospace;">fixed-top</span>: freezes the navbar at the top of the browser window</li>
<li><span style="font-family: "courier new" , "courier" , monospace;">navbar-expand</span>: puts the navbar elements in a horizontal line</li>
<li><span style="font-family: "courier new" , "courier" , monospace;">navbar-dark</span>: makes the navbar dark themed</li>
<li><span style="font-family: "courier new" , "courier" , monospace;">bg-dark</span>: makes the background dark and goes with the dark theme, otherwise the navbar would be invisible</li>
</ul>
<div>
With those general navbar options set up, the next tag is a link with the class <span style="font-family: "courier new" , "courier" , monospace;">navbar-brand</span>. This link sets up a page logo that you can click on to go to the home page, and it's set off from the other links. Next, we have an unordered list with the class <span style="font-family: "courier new" , "courier" , monospace;">navbar-nav</span>, which applies formatting for this to be a list of navigation links. Each list item gets a class of <span style="font-family: "courier new" , "courier" , monospace;">nav-item</span> and each link gets a class of <span style="font-family: "courier new" , "courier" , monospace;">nav-link</span> so the proper formatting can be applied to those elements, too. Now we can take a look at our nice new navigation bar:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEga5vjfapbv5aWwZ4IOaOXzSsJoKfDBLrfFvHM_xwlQSSsHn9Oo6D2g80bK8thTlLVyQIxE6adxZwEgMiZ-NQ0M6_k8_RpOXLxPPqP-EkkVLnXEPA4ohsXjL1SCRUfaZKUmcR6kscPzvxo/s1600/navbar_bad_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot with overlapping navbar" border="0" data-original-height="410" data-original-width="819" height="198" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEga5vjfapbv5aWwZ4IOaOXzSsJoKfDBLrfFvHM_xwlQSSsHn9Oo6D2g80bK8thTlLVyQIxE6adxZwEgMiZ-NQ0M6_k8_RpOXLxPPqP-EkkVLnXEPA4ohsXjL1SCRUfaZKUmcR6kscPzvxo/s400/navbar_bad_screenshot.png" title="" width="400" /></a></div>
<br />
Oops, that doesn't look quite right. The navbar is covering up some of our page because it's a fixed element at the top of the page. We need to drop the rest of the page down a bit, and the easiest way to do that is to add some padding to the <span style="font-family: "courier new" , "courier" , monospace;"><body></span> tag in <span style="font-family: "courier new" , "courier" , monospace;">app/assets/stylesheets/global.scss</span>:<br />
<pre><code class="css">body {
padding-top: 4.5rem;
}</code></pre>
Even though this was a newly created file, any .scss file that appears in <span style="font-family: "courier new" , "courier" , monospace;">app/assets/stylesheets/</span> will get merged into one CSS file that's loaded for every page on the site. It's a Rails convention to put page-specific CSS in their own files for each page, even though it all gets merged together, so that's what we'll do. The home page should look better now:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjockXlh39qh3RqBbbppUmY9dckRIVN372mMrCDN1ZdUsgJDQI75sqHfVmNAlJBs5R4MetxUxmSLbTj1Sk_PlGifl8ogfW64m9sY5gyuKxOannymsteATwhW74iKlkhyVMp-RGYGqiMqxg/s1600/navbar_good_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of fixed non-overlapping navbar" border="0" data-original-height="499" data-original-width="819" height="242" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjockXlh39qh3RqBbbppUmY9dckRIVN372mMrCDN1ZdUsgJDQI75sqHfVmNAlJBs5R4MetxUxmSLbTj1Sk_PlGifl8ogfW64m9sY5gyuKxOannymsteATwhW74iKlkhyVMp-RGYGqiMqxg/s400/navbar_good_screenshot.png" title="" width="400" /></a></div>
<br />
We can also verify that the same navbar appears on the other pages:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHnyof2xS6vHB-PUcoP4C0AYzBbXxvfvGh4GQ_116Er_7SrbWK33yZkD7Zs-P9nbMmERdsaVigkgrD4Xc-pHZHb3dtMSTWDBL6-ANf7crjxHJJDHgbZ_t3hLo9wcSdHUETyPmY4lp6PC8/s1600/navbar_table_screenshot.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Screenshot of navbar on monster materials page" border="0" data-original-height="504" data-original-width="805" height="248" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHnyof2xS6vHB-PUcoP4C0AYzBbXxvfvGh4GQ_116Er_7SrbWK33yZkD7Zs-P9nbMmERdsaVigkgrD4Xc-pHZHb3dtMSTWDBL6-ANf7crjxHJJDHgbZ_t3hLo9wcSdHUETyPmY4lp6PC8/s400/navbar_table_screenshot.png" title="" width="400" /></a></div>
<div>
<br />
Sweet! Navigation is much improved now that we can select any table we want to look at from any page. Those page titles are probably too long to be sustainable as we add more pages, and having a "Home" link in addition to the branding link is redundant; but for now it fills up the navbar nicely. We can condense things later as we need to. What we've gained here is an easy way to add more pages that we can easily navigate to, and it looks pleasing, too. Maybe some color would be nice, but that's easy enough to fiddle with in the CSS at our leisure. Next time, we'll create the big kahuna of the tables: the monster table, and we'll start linking the elements in that table to the other tables we've created.</div><div><br /></div><div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div></div>
Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-14674059233653394102020-05-12T20:17:00.001-05:002020-10-26T20:19:12.095-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2: Viewing DataRails apps are built on an MVC (Model, View, Controller) architecture. In the <a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">last</a> <a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">few</a> <a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">articles</a> of this miniseries, we've focused exclusively on the model component of MVC, building tables in the database, building corresponding models in Rails, and importing the data through Rails models into the database. Now that we have a bunch of monster taming data in the database, we want to be able to look at that data and browse through it in a simple way. We want a view of that data. In order to get that view, we'll need to request data from the model and make it available to the view for display, and that is done through the controller. The view and controller are tightly coupled, so that we can't have a view without the controller to handle the data. We also need to be able to navigate to the view in a browser, which means we'll need to briefly cover routes as well. Since that's quite a bit of stuff to cover, we'll start with the simpler monster material model as a vehicle for explanation.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXr5Eja499eC9xC0C439Nmp4Pnlt-pFNyy0wjQXhUb8vzYHYQ85g8omPmn1G6FBa7222SwP9PZHcJkcbYQMkNaVUWkvtyvmCQmIOA0JcXMqBE19n5fN4Z5ozgnB5lfn-UcellREKiGeFE/s1600/ffxiii2_battle_scene.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Final Fantasy XIII-2 Battle Scene" border="0" data-original-height="288" data-original-width="512" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXr5Eja499eC9xC0C439Nmp4Pnlt-pFNyy0wjQXhUb8vzYHYQ85g8omPmn1G6FBa7222SwP9PZHcJkcbYQMkNaVUWkvtyvmCQmIOA0JcXMqBE19n5fN4Z5ozgnB5lfn-UcellREKiGeFE/s400/ffxiii2_battle_scene.jpg" title="" width="400" /></a></div>
<br />
<a name='more'></a><h4>
Create All The Things</h4>
<script>hljs.initHighlightingOnLoad();</script>
Before we create the view for the monster material model, we'll want to create an index page that will have links to all of the views and different analyses we'll be creating. This index will be a simple, static page so it's an even better place to start than the material view. To create the controller and view for an index page, we enter this in the shell:<br />
<pre><code class="csh">$ rails g controller Home index</code></pre>
This command creates a bunch of files, but most importantly for this discussion it creates <span style="font-family: "courier new" , "courier" , monospace;">app/controllers/home_controller.rb</span> and <span style="font-family: "courier new" , "courier" , monospace;">app/views/home/index.erb</span>. If you haven't guessed by the names, these are our home controller and view for the index page, respectively. The command also creates an entry in <span style="font-family: "courier new" , "courier" , monospace;">config/routes.rb</span> for the route to the index page. We want to add an entry to this file so that going to the root of our website will also take us to the index:<br />
<pre><code class="ruby">Rails.application.routes.draw do
get 'home/index'
root 'home#index'
end</code></pre>
These routes are simple. The first one says if we go to our website (which will be at <span style="font-family: "courier new" , "courier" , monospace;">http://localhost:3000/</span> when we start up the server in a minute), and go to <span style="font-family: "courier new" , "courier" , monospace;">http://localhost:3000/home/index</span>, the HTML in <span style="font-family: "courier new" , "courier" , monospace;">app/views/home/index.erb</span> will be rendered to the browser. The next line says if we go to <span style="font-family: "courier new" , "courier" , monospace;">http://localhost:3000/</span>, that same HTML will be rendered. Currently, that page will show a simple header with the name of the controller and action associated with the page, and the file path to the view:<br />
<pre><code class="html"><h1>Home#index</h1>
<p>Find me in app/views/home/index.html.erb</p></code></pre>
Let's change that to something closer to what we're aiming for:<br />
<pre><code class="html"><h1>Final Fantasy XIII-2 Monster Taming</h1>
<%= link_to 'Monster Materials', '#' %></code></pre>
That second line with the link is created with a special line of code using the '<%= ... %>' designation. This file is not pure HTML, but HAML, an HTML templating language. The '<%= … %>' tag actually means that whatever's inside it should be executed as code and the output is put in its place as HTML. The <span style="font-family: "courier new" , "courier" , monospace;">link_to</span> function is a Rails function that creates the HTML for a link with the given parameters. Now we have a proper title and the first link to a table of data that doesn't exist. That's why I used the '#' character for the link. It tells Rails that there should be a link here, but we don't know what it is, yet. More precisely, Rails will ignore the '#' at the end of a URL, so the link will show up, but it won't do anything when it's clicked. Now let's build the page that will fill in the endpoint for that link.<br />
<br />
<h4>
Create a Monster Materials Page</h4>
Notice that for the index page we created a controller, but we didn't do anything with it. The boilerplate code created by Rails was sufficient to display the page that we created. For the materials page we'll need to do a little more work because we're going to be displaying data from the material table in the database, and the controller will need to make that data available to the view for display. First thing's first, we need to create the controller in the shell:<br />
<pre><code class="csh">$ rails g controller Material index</code></pre>
This command is identical to the last Rails command, and it creates all of the same files for a material controller and view and adds an entry in <span style="font-family: "courier new" , "courier" , monospace;">config/routes.rb</span> for the new page:<br />
<pre><code class="ruby">Rails.application.routes.draw do
get 'material/index'
get 'home/index'
root 'home#index'
end</code></pre>
In both cases we're creating a controller with only one action, but a Rails controller can have many different actions for creating, reading, updating, and deleting objects from a model. These are referred to as CRUD actions. Since we're only going to be viewing this data, not changing it in any way, we just need the read actions, and more specifically the index action because we're only going to look at the table, not individual records. Therefore, we specified the 'index' action in the generate command so the others wouldn't be created. Now it's time to do something useful with that action in <span style="font-family: "courier new" , "courier" , monospace;">app/controllers/material_controller.rb</span>:<br />
<pre><code class="ruby">class MaterialController < ApplicationController
def index
@materials = Material.all
end
end</code></pre>
All we had to do was add that one line in the index action, and we've made all of the material model data available to the view. The view has access to any instance variables that are assigned in the controller, so <span style="font-family: "courier new" , "courier" , monospace;">@materials</span> contains all the data we need to build a view of the material table. The HTML code to render the view is a bit more complex, but still pretty simple:<br />
<pre><code class="html"><h1>Monster Materials</h1>
<table>
<tr>
<th>Name</th>
<th>Grade</th>
<th>Type</th>
</tr>
<% @materials.each do |material| %>
<tr>
<td><%= material.name %></td>
<td><%= material.grade %></td>
<td><%= material.material_type %></td>
</tr>
<% end %>
</table>
</code></pre>
The first half of this code is normal HTML with the start of a table and a header defined. The rows of table data are done with a little HAML to iterate through every material that we have available in the <span style="font-family: "courier new" , "courier" , monospace;">@materials</span> variable. The line with '<% ... %>' just executes what's within the brackets without outputting anything to render. The lines that specify the table data for each cell with '<%= ... %>' will send whatever output happens—in this case the values of the material properties—to the renderer. We could even create dynamic HTML tags in this embedded code to send to the renderer, if we needed to. Here we were able to create the 40 rows of this table in seven lines of code by looping through each material and sending out the property values to the table. This tool is simple, but powerful.<br />
<br />
Now we have another page with a table of monster materials, but we can only reach it by typing the correct path into the address bar. We need to update the link on our index page:<br />
<pre><code class="html"><h1>Final Fantasy XIII-2 Monster Taming</h1>
<%= link_to 'Monster Materials', material_index_path %></code></pre>
It's as simple as using the provided helper function for that route! Rails creates variables for every route defined in <span style="font-family: "courier new" , "courier" , monospace;">config/routes.rb</span> along with a bunch of default routes for other things that we won't get into. We can see these routes by running "<span style="font-family: "courier new" , "courier" , monospace;">rails routes</span>" in the shell, or navigating to <span style="font-family: "courier new" , "courier" , monospace;">/routes</span> on the website. Actually, trying to navigate to any route that doesn't exist will show the routes and their helper functions, which is what happens when we try to get to <span style="font-family: "courier new" , "courier" , monospace;">/routes</span>, too. How convenient. Now we can get to the monster material table from the main index, and amazingly, the table is sorted the same way it was when we imported it. It's pretty plain, though.<br />
<br />
<h4>
Adding Some Polish</h4>
The material table view is functional, but it would be nicer to look at if it wasn't so...boring. We can add some polish with the popular front-end library, <a href="https://getbootstrap.com/">Bootstrap</a>. There are numerous other more fully featured, more complicated front-end libraries out there, but Bootstrap is clean and easy so that's what we're using. We're going to need to install a few gems and make some other changes to config files to get everything set up. To make matters more complicated, the instructions on the <a href="https://github.com/twbs/bootstrap-rubygem" target="_blank">GitHub Bootstrap Ruby Gem page</a> are for Rails 5 using Bundler, but Rails 6 uses Webpacker, which works a bit differently. I'll quickly summarize the steps to run through to get <a href="https://www.timdisab.com/installing-bootstrap-4-on-rails-6/" target="_blank">Bootstrap installed in Rails 6 from this nice tutorial</a>.<br />
<br />
First, use yarn to install Bootstrap, jQuery, and Popper.js:<br />
<pre><code class="csh">$ yarn add bootstrap jquery popper.js</code></pre>
Next, add Bootstrap to the Rails environment by adding the middle section of the following snippet to <span style="font-family: "courier new" , "courier" , monospace;">config/webpack/environment.js</span> between the existing top and bottom lines:<br />
<pre><code class="javascript">const { environment } = require('@rails/webpacker')
const webpack = require('webpack')
environment.plugins.append('Provide',
new webpack.ProvidePlugin({
$: 'jquery',
jQuery: 'jquery',
Popper: ['popper.js', 'default']
})
)
module.exports = environment</code></pre>
Then, set up Bootstrap to start with Rails in app/javascript/packs/application.js by adding this snippet after the require statements:<br />
<pre><code class="javascript">import "bootstrap";
import "../stylesheets/application";
document.addEventListener("turbolinks:load", () => {
$('[data-toggle="tooltip"]').tooltip()
$('[data-toggle="popover"]').popover()
})
</code></pre>
We may never need the tooltip and popover event listeners, but we'll add them just in case. As for that second import statement, we need to create that file under <span style="font-family: "courier new" , "courier" , monospace;">app/javascript/stylesheets/application.scss</span> with this lonely line:<br />
<pre><code class="ruby">@import "~bootstrap/scss/bootstrap";</code></pre>
Finally, we need to add a line to <span style="font-family: "courier new" , "courier" , monospace;">app/views/layouts/application.html.erb</span> for a <span style="font-family: "courier new" , "courier" , monospace;">stylesheet_pack_tag</span>:<br />
<pre><code class="html"><!DOCTYPE html>
<html>
<head>
<title>Bootstrapper</title>
<%= csrf_meta_tags %>
<%= csp_meta_tag %>
<%= stylesheet_link_tag 'application', media: 'all', 'data-turbolinks-track': 'reload' %>
<%= stylesheet_pack_tag 'application', media: 'all', 'data-turbolinks-track': 'reload' %>
<%= javascript_pack_tag 'application', 'data-turbolinks-track': 'reload' %>
</head>
<body>
<%= yield %>
</body>
</html></code></pre>
Whew. Now, we can restart the Rails server, reload the Monster Material page…and see that all that really happened was the fonts changed a little.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKd5By-2UtH3j7S4t5yD5CIolTOWwsJLzPMyk9bKJDxWGRdmMa8OMAjBBZ0CTGUg_gsm3MBUcGhC9hnr6tCX92nsV-xSK7HfexEaSP4KLY7ebqiNu1_j7SeTUbporD74fT2nheyR2OYE4/s1600/monster_materials_table1.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" data-original-height="407" data-original-width="342" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKd5By-2UtH3j7S4t5yD5CIolTOWwsJLzPMyk9bKJDxWGRdmMa8OMAjBBZ0CTGUg_gsm3MBUcGhC9hnr6tCX92nsV-xSK7HfexEaSP4KLY7ebqiNu1_j7SeTUbporD74fT2nheyR2OYE4/s1600/monster_materials_table1.png" title="Plain monster material table" /></a></div>
<br />
Still boring. That's okay. It's time to start experimenting with Bootstrap classes so we can prettify this table. Bootstrap has some incredibly clear documentation for us to select the look that we want. All we have to do is add classes to various elements in <span style="font-family: "courier new" , "courier" , monospace;">app/views/material/index.html.erb</span>. The <span style="font-family: "courier new" , "courier" , monospace;">.table</span> class is a must, and I also like the dark header row, the striped table, and the smaller rows, so let's add those classes to the <span style="font-family: "courier new" , "courier" , monospace;">table</span> and <span style="font-family: "courier new" , "courier" , monospace;">thead</span> elements:<br />
<pre><code class="html"><h1>Monster Materials</h1>
<table id="material-table" class="table table-striped table-sm">
<thead class="thead-dark">
<tr>
<th scope="col">Name</th>
<th scope="col">Grade</th>
<th scope="col">Type</th>
</tr>
</thead></code></pre>
I added an id to the table as well so that we can specify additional properties in <span style="font-family: "courier new" , "courier" , monospace;">app/assets/stylesheets/material.scss</span> because as it is, Bootstrap stretches this table all the way across the page. We can fix that by specifying a width in the .scss file using the new id, and since we're in there, why don't we add a bit of margin for the header and table, too:<br />
<pre><code class="css">h1 {
margin-left: 5px;
}
#material-table {
width: 350px;
margin-left: 5px;
}</code></pre>
We end up with a nice, clean table to look at:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigxAtBaxswaUYBidl8FQW-HP_7Rc6bpAUTibJCtCpvIsdIGe2BMylND-oWR_ESfZwfms7wpU0s1REFfly__qlVmhuVNtQaJtmZVCJh8FOsBPKhv7oVO6xyXh68SjMreGMEXkupD66pQeo/s1600/monster_material_table2.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" data-original-height="495" data-original-width="359" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigxAtBaxswaUYBidl8FQW-HP_7Rc6bpAUTibJCtCpvIsdIGe2BMylND-oWR_ESfZwfms7wpU0s1REFfly__qlVmhuVNtQaJtmZVCJh8FOsBPKhv7oVO6xyXh68SjMreGMEXkupD66pQeo/s1600/monster_material_table2.png" title="Monster material table with Bootstrap skin" /></a></div>
<br />
Isn't that slick? In fairly short order, we were able to set up an index page and our first table page view of monster materials, and we made the table look fairly decent. We have five more tables to go, and some of them are a bit more complicated than this one, to say the least. Our site navigation is also somewhere between clunky and non-existent. We'll make progress on both tables and navigation next time.<div><br /></div><div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div></div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-28829250052757157632020-04-20T20:22:00.001-05:002020-10-26T20:19:01.008-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2: The Remaining TablesContinuing this miniseries of exploring the monster taming mechanics of Final Fantasy XIII-2, we'll finish off the remaining database tables that we want for the data relevant to monster taming. <a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">In the last article</a>, we filled in a second table of passive abilities and connected those abilities to the monsters that had them through references in the monster table that used foreign keys to the ability table. <a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">In the first article</a>, we had identified four tables besides the monster table that we would need as well, these being abilities, game areas, monster materials, and monster characteristics. We did the passive abilities table, but we still need a table for role abilities. In addition to this role ability table, we'll finish off the game areas, monster materials, and monster characteristics tables. These tables are all small, so we should be able to get through them without much effort.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbXc-b2_fXQJv_G-EJzzOKBMtsGAMIXAm_SDiSA9Qzymzd5FDXIDYXCYdbzC-jDMtMu4GSbq9QzZ4TgYBbFR76VIgVGfuu1V180OsaeqOnG1BYy0MRGRRUrzfVHwlFgNprxASaXwOYCj8/s1600/FFXIII-2-Battle-Scene-2.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Final Fantasy XIII-2 Battle Scene" border="0" data-original-height="562" data-original-width="1000" height="223" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbXc-b2_fXQJv_G-EJzzOKBMtsGAMIXAm_SDiSA9Qzymzd5FDXIDYXCYdbzC-jDMtMu4GSbq9QzZ4TgYBbFR76VIgVGfuu1V180OsaeqOnG1BYy0MRGRRUrzfVHwlFgNprxASaXwOYCj8/s400/FFXIII-2-Battle-Scene-2.jpg" title="" width="400" /></a></div>
<br />
<a name='more'></a><h4>
The Monster Role Abilities</h4>
Unlike the passive monster abilities, of which there were 218, we only have 57 role abilities to worry about, and they can be found in <a href="https://gamefaqs.gamespot.com/ps3/619315-final-fantasy-xiii-2/faqs/63854?page=5">this FAQ from Krystal109</a> on <a href="http://gamefaqs.com/">GameFAQs.com</a>. Since these abilities are already in two HTML tables, it's easy to copy-and-paste them directly into a spreadsheet and tweak it the way we want it. I used Google sheets, and removed the "Best Infusion Source," "Learned by," and "Comments" columns because we don't need them. Then I split the merged cells in the "Role" column and copied the role names as necessary. Finally, I added a column for whether the ability is infusable or not, depending on which of the two tables it came from, before exporting the sheet to a .csv file.<br />
<br />
<script>hljs.initHighlightingOnLoad();</script>
But wait! There are actually 70 infusable role abilities in the <a href="https://gamefaqs.gamespot.com/ps3/619315-final-fantasy-xiii-2/faqs/63731">Monster Infusion FAQ</a> that we've been using, plus the non-infusable role abilities from the second HTML table, so we're missing a few. We are dealing with the common occurrence of incomplete data here. I had to cross-check the lists and add in the missing role abilities from the Monster Infusion FAQ. I ended up with 87 abilities in the end. This manual process is still probably faster than writing a script, but we're not sure, yet, if we have all of the non-infusable role abilities. We'll find out in a minute when we try to link up this data with the monster table.<br />
<br />
Before we can add these associations, we have to generate a model of the role abilities table in Rails, like so:<br />
<pre><code class="csh">$ rails g model RoleAbility name:string role:string infusable:boolean</code></pre>
This creates a migration that's all ready to run:<br />
<pre><code class="csh">class CreateRoleAbilities < ActiveRecord::Migration[6.0]
def change
create_table :role_abilities do |t|
t.string :name
t.string :role
t.boolean :infusable
t.timestamps
end
end
end</code></pre>
Now we can add the import of the role abilities to the <span style="font-family: "courier new" , "courier" , monospace;">seeds.rb</span> script right after the passive abilities import:<br />
<pre><code class="ruby"># passive abilities import from monster_abilities.csv
# ...
csv_file_path = 'db/monster_role_abilities.csv'
CSV.foreach(csv_file_path, {headers: true}) do |row|
role_ability = row.to_hash
role_ability['infusable'] = (role_ability['infusable'] == 'true')
RoleAbility.create!(role_ability)
puts "#{row['name']} added!"
end
# monster data import from monsters.csv
# ...</code></pre>
This import looks almost exactly like the passive abilities import, except that we need to convert the 'true' and 'false' strings for the infusable attribute into boolean values to match that attribute's data type. Once we have the role abilities imported and available, we need to add the associations to the monster table. We do this in a very similar way to how we did the passive ability associations, but this time it's simpler because we don't have to worry about red-locked abilities with the same base name as other abilities or ranks that complicated the passive abilities. We just have to find each role ability in each monster's set of skills and associate them by assigning the role ability to the right skill, like so:<br />
<pre><code class="ruby">CSV.foreach(csv_file_path, {headers: true}) do |row|
monster = row.to_hash
# ... associate passive abilities ...
monster.keys.select { |key| key.include? '_skill' }.each do |key|
if monster[key]
monster[key] = RoleAbility.find_by(name: monster[key])
if monster[key].nil?
puts "ERROR: monster #{monster['name']} #{key} not found!"
return
end
puts "Found #{key} #{monster[key].name}"
end
end
Monster.create!(monster)
puts "#{row['name']} added!"
end</code></pre>
Now, we can't run this seed script quite yet. We still don't have a database migration or the monster and role ability models set up to handle these associations, so let's do that tedious work. First, the monster table migration needs all of the skill attributes changed from <span style="font-family: "courier new" , "courier" , monospace;">t.string</span> to <span style="font-family: "courier new" , "courier" , monospace;">t.references</span> and <span style="font-family: "courier new" , "courier" , monospace;">add_foreign_key</span> statements need to be added for each skill. There are about a hundred of these, so I'll just show an example of what they look like:<br />
<pre><code class="ruby">class CreateMonsters < ActiveRecord::Migration[6.0]
def change
create_table :monsters do |t|
# ... all of the other monster attributes ...
t.references :default_skill1
t.references :default_skill2
t.references :default_skill3
t.references :default_skill4
t.references :default_skill5
t.references :default_skill6
t.references :default_skill7
t.references :default_skill8
t.references :lv_02_skill
t.references :lv_03_skill
# ... etc ...
end
# ... passive ability foreign keys ...
add_foreign_key :monsters, :role_abilities, column: :default_skill1_id, primary_key: :id
add_foreign_key :monsters, :role_abilities, column: :default_skill2_id, primary_key: :id
add_foreign_key :monsters, :role_abilities, column: :default_skill3_id, primary_key: :id
add_foreign_key :monsters, :role_abilities, column: :default_skill4_id, primary_key: :id
add_foreign_key :monsters, :role_abilities, column: :default_skill5_id, primary_key: :id
add_foreign_key :monsters, :role_abilities, column: :default_skill6_id, primary_key: :id
add_foreign_key :monsters, :role_abilities, column: :default_skill7_id, primary_key: :id
add_foreign_key :monsters, :role_abilities, column: :default_skill8_id, primary_key: :id
add_foreign_key :monsters, :role_abilities, column: :lv_02_skill_id, primary_key: :id
add_foreign_key :monsters, :role_abilities, column: :lv_03_skill_id, primary_key: :id
# ... etc ...
end
end</code></pre>
With that done, we still need to add the role abilities to the monster model in <span style="font-family: "courier new" , "courier" , monospace;">app/models/monster.rb</span> with <span style="font-family: "courier new" , "courier" , monospace;">belongs_to</span>. Remember how it felt weird that we were saying a monster belongs to its abilities in the last article? Well, that's still the direction we want, so we'll do it again with role abilities, even though it sounds weird:<br />
<pre><code class="ruby">class Monster < ApplicationRecord
# ... passive ability belongs_to declarations
belongs_to :default_skill1, class_name: 'RoleAbility', optional: true
belongs_to :default_skill2, class_name: 'RoleAbility', optional: true
belongs_to :default_skill3, class_name: 'RoleAbility', optional: true
belongs_to :default_skill4, class_name: 'RoleAbility', optional: true
belongs_to :default_skill5, class_name: 'RoleAbility', optional: true
belongs_to :default_skill6, class_name: 'RoleAbility', optional: true
belongs_to :default_skill7, class_name: 'RoleAbility', optional: true
belongs_to :default_skill8, class_name: 'RoleAbility', optional: true
belongs_to :lv_02_skill, class_name: 'RoleAbility', optional: true
belongs_to :lv_03_skill, class_name: 'RoleAbility', optional: true
# ... etc ...
end</code></pre>
Finally, we need to add the has_many declarations to the role ability model (because each ability has many monsters, right?):<br />
<pre><code class="ruby">class RoleAbility < ApplicationRecord
has_many :default_skill1_monsters, :class_name => 'Monster', :foreign_key => 'default_skill1'
has_many :default_skill2_monsters, :class_name => 'Monster', :foreign_key => 'default_skill2'
has_many :default_skill3_monsters, :class_name => 'Monster', :foreign_key => 'default_skill3'
has_many :default_skill4_monsters, :class_name => 'Monster', :foreign_key => 'default_skill4'
has_many :default_skill5_monsters, :class_name => 'Monster', :foreign_key => 'default_skill5'
has_many :default_skill6_monsters, :class_name => 'Monster', :foreign_key => 'default_skill6'
has_many :default_skill7_monsters, :class_name => 'Monster', :foreign_key => 'default_skill7'
has_many :default_skill8_monsters, :class_name => 'Monster', :foreign_key => 'default_skill8'
has_many :lv_02_skill_monsters, :class_name => 'Monster', :foreign_key => 'lv_02_skill'
has_many :lv_03_skill_monsters, :class_name => 'Monster', :foreign_key => 'lv_03_skill'
# ... etc ...
end</code></pre>
Whew! We're finally ready to purge the database, run the migration again, and reseed the database, but remember to rename the monster table migration so that the timestamp in the filename is after the new role ability migration because the monster table migration needs to run last:<br />
<pre><code class="csh">$ mv <old monster migration name> <new monster migration name>
$ rails db:purge
$ rails db:migrate
$ rails db:seed</code></pre>
After running these commands, we find that a bunch of non-infusable role abilities were missing from the list. Most of these were basic Commando commands like Attack, Ruin, Launch, and Blitz, or variations on Saboteur commands like Dispel II or Heavy Dispelga. When all missing abilities are found and fixed, we should have 127 role abilities. Then there are two typos in the Monster Infusion FAQ monster list that need to be fixed as well: an instance of "Deprotectga" instead of "Deprotega" and an instance of "Medigaurd" instead of "Mediguard." With that task done, we can call the role ability table complete. There are six hidden role abilities, one for each role type, but they are never referenced in the monster attributes because they are fixed by the role type of the monster. By infusing 99 levels worth of monsters of the opposite role type, (e.g. commando and ravager are opposites) the monster will acquire its hidden role ability. It's questionable whether adding them to the database is useful, so let's move on to game areas.<br />
<br />
<h4>
Making a Game Location Graph</h4>
You start the game in New Bodhum 003 AF. Actually, you start in Valhalla, but after getting through the intro segment with Lightning and a bunch of over-the-top cut scenes, the game really starts in New Bodhum with Serah. As you progress through the game, you unlock more and more new areas by jumping through time gates. Some locations have multiple time gates that lead to multiple new locations. Different monsters live in different locations, and which locations they live in is noted as one or more of the monsters' location attributes.<br />
<br />
We want to capture the graph of these location dependencies in a database table and link the locations to the monster attributes so that we can figure out the earliest possible times in the game that we can acquire monsters with certain abilities. In order to do that, we're going to build a table to represent that graph. We could build this table in a couple of ways. One way is to make an adjacency matrix, but this representation requires a row and column for each node in the graph. In this case such a matrix would be sparsely populated, so we're going to use an adjacency list instead. Each row in the table will have one location's name, and which location leads to this location. Since each location has only one other location as a source, we can get away with this simple table structure. There are only a small number of locations, so the .csv file for this table can be created by hand. Plus, I couldn't find a good list on the internet, so here it is, created from scratch:<br />
<pre><code class="text">name,source
New Bodhum 003 AF,nil
Bresha Ruins 005 AF,New Bodhum 003 AF
Bresha Ruins 300 AF,Bresha Ruins 005 AF
Yaschas Massif 110 AF,Bresha Ruins 300 AF
Yaschas Massif 010 AF,Bresha Ruins 005 AF
Oerba 200 AF,Yaschas Massif 010 AF
Yaschas Massif 01X AF,Oerba 200 AF
Augusta Tower 300 AF,Yaschas Massif 01X AF
Serendipity Year Unknown,Yaschas Massif 01X AF
Sunleth Waterscape 300 AF,Bresha Ruins 005 AF
Coliseum ??? AF,Sunleth Waterscape 300 AF
Archylte Steppe ??? AF,Sunleth Waterscape 300 AF
Vile Peaks 200 AF,Archylte Steppe ??? AF
Academia 400 AF,Sunleth Waterscape 300 AF
Yaschas Massif 100 AF,Academia 400 AF
Sunleth Waterscape 400 AF,Yaschas Massif 100 AF
Augusta Tower 200 AF,Academia 400 AF
Oerba 300 AF,Augusta Tower 200 AF
Oerba 400 AF,Oerba 300 AF
Academia 4XX AF,Augusta Tower 200 AF
The Void Beyond ??? AF,Academia 4XX AF
Vile Peaks 010 AF,Academia 4XX AF
A Dying World 700 AF,Academia 4XX AF
Bresha Ruins 100 AF,A Dying World 700 AF
New Bodhum 700 AF,A Dying World 700 AF
Academia 500 AF,New Bodhum 700 AF
Valhalla ??? AF,Academia 500 AF
Coliseum ??? AF (DLC),New Bodhum 003 AF
Valhalla ??? AF (DLC),New Bodhum 003 AF
Serendipity ??? AF (DLC),New Bodhum 003 AF</code></pre>
As we've already done with the other tables, the first thing we need to do to import this .csv file is generate the model for this location table:<br />
<pre><code class="csh">$ rails g model Location name:string source:references</code></pre>
Notice that we created the source attribute as a reference right away, and this reference that's being created is different than the others that we created for passive and role abilities. Whereas those references were associated with new tables, this source reference is associated with the location table itself. To construct this self-reference, we don't actually have to do anything to the migration. Rails does the right thing by default, so the migration is simple:<br />
<pre><code class="ruby">class CreateLocations < ActiveRecord::Migration[6.0]
def change
create_table :locations do |t|
t.string :name
t.references :source
t.timestamps
end
end
end</code></pre>
We also want to add references to the location attributes in the monster table, so we need to change that migration as well:<br />
<pre><code class="ruby">class CreateMonsters < ActiveRecord::Migration[6.0]
def change
create_table :monsters do |t|
t.string :name
t.string :role
t.references :location, foreign_key: true
t.references :location2, foreign_key: {to_table: :locations}
t.references :location3, foreign_key: {to_table: :locations}
# ... the mess of other attribute declarations ...
end
# ... a mess of foreign key declarations ...
end
end
</code></pre>
Notice that for these location references, the foreign key is specified directly in the attribute declaration. I recently noticed that this was possible, and it's much cleaner than adding foreign keys at the end, so I'm switching to doing it this way.<br />
<br />
The next step is to update the <span style="font-family: "courier new" , "courier" , monospace;">seeds.rb</span> script for the location and monster models. Let's do the models first this time. The location model will need a self-reference for the <span style="font-family: "courier new" , "courier" , monospace;">source</span> attribute and declare that it has many monsters for the location attributes in the monster model, like so:<br />
<pre><code class="ruby">class Location < ApplicationRecord
belongs_to :source, class_name: 'Location', optional: true
has_many :location_monsters, :class_name => 'Monster', :foreign_key => 'location'
has_many :location2_monsters, :class_name => 'Monster', :foreign_key => 'location2'
has_many :location3_monsters, :class_name => 'Monster', :foreign_key => 'location3'
end</code></pre>
The self-reference doesn't need to declare its foreign key because it's the same name as the name of the reference. Rails can infer the foreign key correctly in this case. Now for the monster model:<br />
<pre><code class="ruby">class Monster < ApplicationRecord
belongs_to :location, class_name: 'Location', optional: true
belongs_to :location2, class_name: 'Location', optional: true
belongs_to :location3, class_name: 'Location', optional: true
# ... a mess of other reference declarations ...
end</code></pre>
It makes a little more sense that a monster belongs to a location, so that's nice, but it's really just words. It's the direction of the associations that's important, and <span style="font-family: "courier new" , "courier" , monospace;">belongs_to</span> means the reference is in the current model and points to the Location model, which is what we want. Finally, we can update the <span style="font-family: "courier new" , "courier" , monospace;">seeds.rb</span> script to import locations. This update happens in two parts. First, we need to read from the .csv file we created:<br />
<pre><code class="ruby"># ... role abilities import from monster_role_abilities.csv ...
csv_file_path = 'db/monster_locations.csv'
CSV.foreach(csv_file_path, {headers: true}) do |row|
location = row.to_hash
location['source'] = Location.find_by(name: location['source'])
if location['source'].nil? && location['name'] != 'New Bodhum 003 AF'
puts "ERROR: location #{location['source']} not found!"
return
end
Location.create!(location)
puts "Location #{location['name']} added!"
end
# ... monster data import from monsters.csv ...</code></pre>
For this import, we're not only populating the table, but we have to correctly assign the self-references. To do that assignment, we had to make sure when writing the .csv file that every location that was referenced in a source attribute appeared before that reference as a named location. That way, when this script looks for a location in the source attribute, it will find it already exists as a location in the table. If there were loops, this import would require two passes to find all of the location names first before assigning references, but there are no loops so lucky us. I also check that each source location is actually found, in case I made any typos, but there is one intentional nil source for where you start the game in New Bodhum 003 AF. That particular nil source needs to be handled specially. With the location table filled in, we can associate the location attributes in the monster table:<br />
<pre><code class="ruby">CSV.foreach(csv_file_path, {headers: true}) do |row|
monster = row.to_hash
# ... associate role abilities ...
monster.keys.select { |key| key.starts_with? 'location' }.each do |key|
if monster[key]
monster[key] = Location.find_by(name: monster[key])
if monster[key].nil?
puts "ERROR: monster #{monster['name']} #{key} not found!"
return
end
puts "Found #{key} '#{monster[key].name}'"
end
end
Monster.create!(monster)
puts "#{row['name']} added!"
end</code></pre>
This part of the script is exactly the same as it was for role abilities, just doing it for the location attributes instead. Now we can run this script again, making sure that the monster migration is still the most recent migration since it needs to be performed last:<br />
<pre><code class="csh">$ mv <old monster migration name> <new monster migration name>
$ rails db:purge
$ rails db:migrate
$ rails db:seed</code></pre>
After running this script, I found a number of additional typos in the FAQ, so that data validation code is really coming in handy. I also caught an unexpected shortcut that the FAQ author took with locations. If a monster appears in two similar locations, they combined the years with a slash, e.g. "Bresha Ruins 100/300 AF." I had to split these out into two separate lines to adhere to our data model. Once that shortcut and the typos were fixed, the script ran to completion, and that's it for locations.<br />
<br />
<h4>
Wrapping up Monster Materials and Characteristics</h4>
This article is getting pretty long, so I'm going to finish off the monster materials and characteristics tables quickly. The characteristics is a straightforward table that doesn't require figuring out anything new. The materials table can also be straightforward, at least for now. The monster materials themselves constitute a simple table with a name, a grade of 1-5, and a type of either biological or mechanical. The trick is that we'll eventually want to know when in the game we can get each grade of monster material because they're used to upgrade our tamed monsters. The way to acquire those materials is by defeating other monsters in battle. The different materials are some of the loot that the monsters drop. This information is all in the Monster Infusion FAQ, but it would require another parser to extract it and import it into the monster table. We can do that later in the series, but right now I want to complete the tables and get on with starting to view the data so we'll just go with making the simple material table.<br />
<br />
A table of the materials can be found in the same <a href="https://gamefaqs.gamespot.com/ps3/619315-final-fantasy-xiii-2/faqs/63854?page=5#section729">HTML FAQ from Krystal109</a>. Like the first set of role abilities, we can copy this table into a spreadsheet and then export it to a .csv file. Then we run through the same steps of importing this data into our Rails database, starting with generating a material model:<br />
<pre><code class="csh">$ rails g model Material name:string grade:integer material_type:string</code></pre>
The migration is created without need for modification, and right now, this will be a stand-alone table so there's no need to change any model code. All that's left is to import the .csv file in the <span style="font-family: "courier new" , "courier" , monospace;">seeds.rb</span> script:<br />
<pre><code class="ruby"># ... location import from monster_locations.csv ...
csv_file_path = 'db/monster_materials.csv'
CSV.foreach(csv_file_path, {headers: true}) do |row|
Material.create!(row.to_hash)
puts "Material #{row['name']} added!"
end
# ... monster data import from monsters.csv ...</code></pre>
Then we can do the same thing with the monster characteristics table also found in the <a href="https://gamefaqs.gamespot.com/ps3/619315-final-fantasy-xiii-2/faqs/63854?page=5#section713">HTML FAQ from Krystal109</a>. Do the same drill of copying the table into a spreadsheet and then exporting it to a .csv file before generating another model for this table:<br />
<pre><code class="csh">$ rails g model Characteristic name:string description:string</code></pre>
Finally, add this data import to the <span style="font-family: "courier new" , "courier" , monospace;">seeds.rb</span> script:<br />
<pre><code class="ruby"># ... material import from monster_materials.csv ...
csv_file_path = 'db/monster_characteristics.csv'
CSV.foreach(csv_file_path, {headers: true}) do |row|
Characteristic.create!(row.to_hash)
puts "Characteristic #{row['name']} added!"
end
# ... monster data import from monsters.csv ...</code></pre>
And we're ready to run the migration and import:<br />
<pre><code class="csh">$ rails db:purge
$ rails db:migrate
$ rails db:seed</code></pre>
The purge is done first to clean out everything that we already had in the database. Otherwise, when we run it again now, we'll be duplicating all of the other data that we already imported in the <span style="font-family: "courier new" , "courier" , monospace;">seeds.rb</span> script on previous runs. Running this sequence of commands will build a fresh database from scratch, and it should finish cleanly in one shot.<br />
<br />
That last step will wrap up the database design and building, for now. We'll have a bit of work to do later when we want to connect the monster materials to the monsters that drop them, but this is good enough to get started with creating views of this data. After spending multiple articles parsing, exporting, importing, and creating data for the database, I'm anxious to get moving on those views, so that's what we'll start looking at next time.<div><br /></div><div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div></div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-37847492815060627342020-03-30T20:26:00.001-05:002020-10-26T20:18:40.325-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2: Relational DataIn this next installment of the miniseries of exploring the monster taming mechanics of Final Fantasy XIII-2, we'll fill out another database table that we need in order to start connecting all of the monster data together. <a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">In the last article</a>, we built the core monster table with hundreds of attributes for each of 164 monsters. <a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">In the first article</a>, we had identified four other tables that we would need as well, these being abilities, game areas, monster materials, and monster characteristics. The data in these four tables is all related in one way or another to the monsters in the monster table. We'll start with the abilities table, which will end up being three tables because we actually have passive, command, and role abilities. Once the passive abilities table is complete, we'll see how to connect that data in the database so that we can later make inferences on the data.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNLfE6YTfQqFZZoK0wfphRo6nP62LM_TZI1w1a1ef4r4PMjhj4P_5Mxl6p8vFJhR3p15SgIAjF2UDewAB4NFP9e4JgGDWdj4HDnvtYkeiFrRaELwr_wa3_hM5lc9cs1uJEWVupQ-KAJ4E/s1600/FFXIII-2-Battle-Scene.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="Final Fantasy XIII-2 battle scene" border="0" data-original-height="338" data-original-width="600" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNLfE6YTfQqFZZoK0wfphRo6nP62LM_TZI1w1a1ef4r4PMjhj4P_5Mxl6p8vFJhR3p15SgIAjF2UDewAB4NFP9e4JgGDWdj4HDnvtYkeiFrRaELwr_wa3_hM5lc9cs1uJEWVupQ-KAJ4E/s400/FFXIII-2-Battle-Scene.jpg" title="" width="400" /></a></div>
<br />
<a name='more'></a><h4>
Finding Monster Abilities</h4>
Like for the core monster data, the first thing we need to do is get the data for the monster abilities into a .csv format. We'll start with the passive abilities because this is the largest group of abilities and they have a rank attribute that the role and hidden abilities don't have. Since there are over 200 passive abilities, we're going to want to pull this data out of the FAQ with another script instead of building it up manually. We can use an FSM very similar to the one used for parsing the monster data with a few simplifications because there are not nearly so many attributes for the passive abilities. I'll just throw the script out there, and then we can discuss it:<br />
<script>hljs.initHighlightingOnLoad();</script>
<br />
<pre><code class="ruby">SECTION_TAG = "PassADe"
ABILITY_SEPARATOR = "........................................"
ABILITY_REGEX = /(\w\S+(?:\s[^\s\(]+)*)\s\((RL|\d)\)-*:\s(\w\S+(?:\s\S+)*)/
ABILITY_EXT_REGEX = /^\s+(\S+(?:\s\S+)*)/
end_abilities = lambda do |line, data|
return end_abilities, data
end
new_ability = lambda do |line, data|
props = line.scan(ABILITY_REGEX)
if props.empty?
if line.include? ABILITY_SEPARATOR
return end_abilities, data
else
extra_line = ABILITY_EXT_REGEX.match(line)
data.last["description"] += ' ' + extra_line[1]
return new_ability, data
end
end
if props.first[1] == "RL"
props.first[1] = "99"
props.first[0] += " (RL)"
end
data << {"name" => props.first[0], "rank" => props.first[1], "description" => props.first[2]}
return new_ability, data
end
find_abilities = lambda do |line, data|
if line.include? ABILITY_SEPARATOR
return new_ability, data
end
return find_abilities, data
end
find_sub_section = lambda do |line, data|
if line.include? SECTION_TAG
return find_abilities, data
end
return find_sub_section, data
end
section_tag_found = lambda do |line, data|
if line.include? SECTION_TAG
return find_sub_section, data
end
return section_tag_found, data
end
start = lambda do |line, data|
if line.include? SECTION_TAG
return section_tag_found, data
end
return start, data
end
next_state = start
data = []
File.foreach("ffiii2_monster_taming_faq.txt") do |line|
next_state, data = next_state.(line, data)
end</code></pre>
Starting from the bottom, we can see that we kickoff the FSM by looking for the <span style="font-family: "courier new" , "courier" , monospace;">SECTION_TAG</span>, which is "PassADe" in this case. We have to find it three times instead of the two times we looked for the section tag in the <a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">monster FSM</a> because the tag appears one extra time in the FAQ in a sub-contents under the main "Monster Infusion" section. Then, we look for the <span style="font-family: "courier new" , "courier" , monospace;">ABILITY_SEPARATOR</span>, which is the same as the <span style="font-family: "courier new" , "courier" , monospace;">MONSTER_SEPARATOR</span> from the previous script, and then go into a loop of matching and storing the ability data in the <span style="font-family: "courier new" , "courier" , monospace;">data</span> list of hash tables. The ability data has a pretty clean format with the possibility of overflowing to a second line. The first couple abilities showcase the variations we have to handle:<br />
<pre><code class="text">Uncapped Damage (RL)---------: Raises the cap on damage to 999,999.
Enhanced Commando (RL)-------: Enhances the Commando Role Bonus by one Bonus
Boost.</code></pre>
We have a name that can be multiple words (and by looking further ahead we also see some special characters) followed by a space and a rank in parentheses, a -: separator, and the description, possibly extending to a second line. We want to capture the name, rank, and description, and the rank can be either "RL" (for Red Lock, meaning it can never be removed from the monster) or a number 1-9. From what we've learned of regexes in <a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">past</a> <a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">posts</a>, it should be clear that the regex<br />
<br />
/(\w\S+(?:\s[^\s\(]+)*)\s\((RL|\d)\)-*:\s(\w\S+(?:\s\S+)*)/<br />
<br />
is sufficient to match on the first line and capture those three attributes. Looking at the rest of the new_ability state code, we can see that if the line doesn't match the <span style="font-family: "courier new" , "courier" , monospace;">ABILITY_REGEX</span>, it checks if the line is the <span style="font-family: "courier new" , "courier" , monospace;">ABILITY_SEPARATOR</span> and moves to the last state if it is. Otherwise, we know that the line is an extra line of description, so we capture it and add it to the description of the last ability captured. If the line did match the <span style="font-family: "courier new" , "courier" , monospace;">ABILITY_REGEX</span>, then we create a new hash with the values populated from the regex captures and continue in the same state.<br />
<br />
Note that if the rank was "RL", we convert that to "99" so that all ranks will be integers in the database. Why not "10?" Well, when an ability is yellow-locked, its rank increases by nine, so making the red-locked abilities have a rank of 10 would conflict. I could have made it 19 or 20, but 99 works just as well and really sets them apart. We also need to add the " (RL)" text back into the name of the ability because it's possible to have red-locked and non-red-locked abilities with the same name, and they are different abilities for all intents and purposes.<br />
<br />
<h4>
Validating and Exporting Monter Abilities</h4>
Next, we want to do a little validation check on this data and write it to a .csv file, so here's the code to do that:<br />
<pre><code class="ruby">PROPER_NAME_REGEX = /^\w.*[\w)!%]$/
NUMBER_REGEX = /^\d\d?$/
FREE_TEXT_REGEX = /^\S+(?:\s\S+)*$/
VALID_ABILITY = {
"name" => PROPER_NAME_REGEX,
"rank" => NUMBER_REGEX,
"description" => FREE_TEXT_REGEX,
}
data.each do |ability|
VALID_ABILITY.each do |key, regex|
if ability.key?(key)
unless ability[key] =~ regex
puts "Monster ability #{ability["name"]} has invalid property #{key}: #{ability[key]}."
end
else
puts "Monster ability #{ability["name"]} has missing property #{key}."
end
end
end
require 'csv'
opts = {headers: data.first.keys, write_headers: true}
CSV.open("monster_abilities.csv", "wb", opts) do |csv|
data.each { |hash| csv << hash }
end</code></pre>
Since we're building up each hash table directly with name, rank, and description, it's not necessary to make sure that each hash contains all three attributes and no others. We just want to make sure that each attribute value has the right (admittedly loose) format. Then we write it out to the monster_abilities.csv file.<br />
<br />
<h4>
Importing Monster Abilities</h4>
Moving right along, we can now import this .csv file into the database with another simple addition to our seed script:<br />
<pre><code class="ruby">csv_file_path = 'db/monster_abilities.csv'
CSV.foreach(csv_file_path, {headers: true}) do |row|
Ability.create!(row.to_hash)
puts "#{row['name']} added!"
end</code></pre>
Before running this script, we need to generate the Ability model and its schema. Since there are only three attributes, we can easily create the migration script in one shot:<br />
<pre><code class="bash">$ rails generate model Ability name:string rank:integer description:string</code></pre>
This command will create this migration file:<br />
<pre><code class="ruby">class CreateAbilities < ActiveRecord::Migration[6.0]
def change
create_table :abilities do |t|
t.string :name
t.integer :rank
t.string :description
t.timestamps
end
end
end</code></pre>
Now we can run the migration and then seed the database:<br />
<pre><code class="bash">$ rails db:migrate
$ rails db:seed</code></pre>
And we've added our second table to the database with 147 abilities. (Wait, I thought there were over 200 abilities. Good catch. We'll get to that in a bit.) However, we should have a link between this new ability table and the monster table because every monster's many abilities should all be one of the abilities in the ability table.<br />
<br />
<h4>
Associating Monsters' Abilities with the Abilities Table</h4>
What we want to do in order to associate each monster's abilities with the abilities table is make each ability in the monster table a reference instead of a string, and point those references at the correct specific abilities in the abilities table. To accomplish this association, we'll need to change both the monster table migration and the monster model part of the seed script.
<br />
<br />
First, we can change all of the <span style="font-family: "courier new" , "courier" , monospace;">t.string</span> declarations in the monster table migration to <span style="font-family: "courier new" , "courier" , monospace;">t.references</span>. Changing only the abilities that end in "_passive" is sufficient for now because those are the abilities we created in the abilities table. The references for these abilities will actually be foreign keys from the abilities table, so we need to tell Rails that at the end of the migration:<br />
<pre><code class="ruby">class CreateMonsters < ActiveRecord::Migration[6.0]
def change
create_table :monsters do |t|
t.string :name
# ... The other monster attributes ...
t.references :default_passive1
t.references :default_passive2
t.references :default_passive3
t.references :default_passive4
t.string :default_skill
t.string :special_notes
t.references :lv_02_passive
t.string :lv_02_skill
# ... The rest of the lv XX abilities ...
t.timestamps
end
add_foreign_key :monsters, :abilities, column: :default_passive1_id, primary_key: :id
add_foreign_key :monsters, :abilities, column: :default_passive2_id, primary_key: :id
add_foreign_key :monsters, :abilities, column: :default_passive3_id, primary_key: :id
add_foreign_key :monsters, :abilities, column: :default_passive4_id, primary_key: :id
add_foreign_key :monsters, :abilities, column: :lv_02_passive_id, primary_key: :id
# ... All of the other _passive abilities foreign keys ...
end
end</code></pre>
I know this seems like tedious busy work, but it's necessary for setting up the database. Some of it can be done with regex replace tools in some editors or generated with a script. I just brute-forced it and got pretty good at changing numbers really rapidly. Be sure to remove lv_xx_passive abilities that don't exist later in the list.<br />
<br />
Because we changed this migration, we need to rerun it, but now it references the ability migration so when we rollback the migrations, we need to change the date in the file name on the monster migration to be after the ability migration before we run the migrations forward again. Otherwise, Rails will complain.<br />
<pre><code class="csh">$ rails db:rollback STEP=2
$ [rename the monster migration file to a date after the ability migration file]
$ rails db:migrate</code></pre>
We're not quite done with preparing the database, yet, because the Rails models need to know about this association, too. The <span style="font-family: "courier new" , "courier" , monospace;">app/models/monster.rb</span> file needs to know that the passive ability attributes are associated with the Ability model using the <span style="font-family: "courier new" , "courier" , monospace;">belongs_to</span> association:<br />
<pre><code class="ruby">class Monster < ApplicationRecord
belongs_to :default_passive1, class_name: 'Ability', optional: true
belongs_to :default_passive2, class_name: 'Ability', optional: true
belongs_to :default_passive3, class_name: 'Ability', optional: true
belongs_to :default_passive4, class_name: 'Ability', optional: true
belongs_to :lv_02_passive, class_name: 'Ability', optional: true
# ... The rest of the lv_XX_passive attributes ...
end</code></pre>
It may sound weird that a monster belongs to an ability, but that's the type of association we want where the link points from the monster attribute to the ability. It's the same as if we had an Author model and a Book model, and the book belongs to the author that wrote it. The link would be in the book with the author's ID. That same link direction is what we want with monsters having links to ability IDs, so <span style="font-family: "courier new" , "courier" , monospace;">belongs_to</span> it is.<br />
<br />
We also need to add associations to the Ability model so that we can follow links from abilities to monsters. That association is done with <span style="font-family: "courier new" , "courier" , monospace;">has_many</span>, which also seems a bit weird, but oh well:<br />
<pre><code class="ruby">class Ability < ApplicationRecord
has_many :default_passive1_monsters, :class_name => 'Monster', :foreign_key => 'default_passive1'
has_many :default_passive2_monsters, :class_name => 'Monster', :foreign_key => 'default_passive2'
has_many :default_passive3_monsters, :class_name => 'Monster', :foreign_key => 'default_passive3'
has_many :default_passive4_monsters, :class_name => 'Monster', :foreign_key => 'default_passive4'
has_many :lv_02_passive_monsters, :class_name => 'Monster', :foreign_key => 'lv_02_passive'
# ... The rest of the lv_XX_passive attributes ...
end</code></pre>
<br />
That was the easy, if mindlessly tedious, part. The next step is trickier. We want to change the db seed script so that when the monsters are created, the passive abilities in the monster table are references to the abilities table, and we want to catch any instances where the ability doesn't exist, meaning there was a typo or an omission. We need to make sure the ability table is populated first in the script, so we have access to those abilities when we're importing the monsters. Then, for each passive ability for each monster we search the ability table for that ability's name, and assign it to the corresponding ability attribute for the monster. That assignment creates the proper reference. If the ability isn't found, we print an error and return so the error can be fixed in the FAQ text file. We'll then have to rerun the ability or monster parser and try the import again. Here's what this process looks like in code, including the ability import:<br />
<pre><code class="ruby">csv_file_path = 'db/monster_abilities.csv'
CSV.foreach(csv_file_path, {headers: true}) do |row|
Ability.create!(row.to_hash)
puts "#{row['name']} added!"
end
csv_file_path = 'db/monsters.csv'
CSV.foreach(csv_file_path, {headers: true}) do |row|
monster = row.to_hash
monster.keys.select { |key| key.ends_with? '_passive' }.each do |key|
if monster[key]
monster[key] = Ability.find_by(name: monster[key])
if monster[key].nil?
puts "ERROR: monster #{monster['name']} #{key} not found!"
return
end
puts "Found #{key} #{monster[key].name}"
end
end
Monster.create!(monster)
puts "#{row['name']} added!"
end</code></pre>
This code is pretty much written as just described, but note that we check if the <span style="font-family: "courier new" , "courier" , monospace;">monster[key]</span> exists before doing anything with it. Remember that these are optional attributes, so they can be <span style="font-family: "courier new" , "courier" , monospace;">nil</span> for any given monster ability. If we run this script, we'll find right away that Cactuaroni has an ability "Critical: Haste (RL)" that doesn't exist in the ability table. We could add this ability to the FAQ and rerun the parser, but notice that there is an ability "Critical: Haste" already in the table. This issue of the non-red-locked ability existing in the table but not the red-locked version is fairly common, so we could solve the problem with code instead of tediously rerunning the scripts to find all of the instances where the red-locked ability is missing. All we have to do is search for the base ability, copy it, change the rank to 99, and tack " (RL)" onto the name. This process can be done like so:<br />
<pre><code class="ruby">CSV.foreach(csv_file_path, {headers: true}) do |row|
monster = row.to_hash
monster.keys.select { |key| key.include? '_passive' }.each do |key|
if monster[key]
if monster[key].ends_with?(' (RL)') && Ability.find_by(name: monster[key]).nil?
ability_name = monster[key][0..-6]
puts "Searching for #{key} ability #{ability_name}"
ability = Ability.find_by(name: ability_name).dup
ability['name'] = monster[key]
ability['rank'] = '99'
ability.save
puts "Ability #{ability['name']} added!"
end
monster[key] = Ability.find_by(name: monster[key])
if monster[key].nil?
puts "ERROR: monster #{monster['name']} #{key} not found!"
return
end
puts "Found #{key} #{monster[key].name}"
end
end
Monster.create!(monster)
puts "#{row['name']} added!"
end</code></pre>
If the monster ability ends with " (RL)" and it's not in the ability table, then we copy the base ability and create the corresponding red-locked ability from it. We've sidestepped a bunch of dreary work with a little bit of targeted code!<br />
<br />
Now after running this script to seed the database, we will find a few more errors in the FAQ to fix. Four red-locked abilities are missing, so we'll have to add Perpetual Poison, Resist Damage +05%, Bonus CP, and Feral Speed. I found definitions for these abilities simply by googling them. The Resist Elements +20% ability that Twilight Odin has was also missing. I guessed at the rank of 8 for this one because I couldn't find it. Resist Elements +30% has a rank of 9 and Resist Elements +05% has a rank of 5, so it's most likely rank 7 or 8. Finally, there were three typos: the ability "Auto: Enfire (RL)" should not have the colon, the ability "Auto Haste (RL)" should be hyphenated, and the ability "ATB: Advantage (RL)" should not have the colon. After everything is fixed, we have a complete ability table with 218 abilities, and every passive monster ability is linked correctly to the ability table.<br />
<br />
<br />
That was quite a lot of work, some of it tedious, but we accomplished a lot. We generated a list of passive abilities from the FAQ, validated that data, imported it into a new table in the database, and linked all of those abilities to the monsters that can learn them in the monster table. This process can be repeated for the much smaller tables that are left to create, and that is what we'll do next time.<div><br /></div><div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div></div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-17116404122627851992020-03-10T21:47:00.001-05:002020-10-26T20:18:28.440-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2: Data Validation and Database ImportContinuing on with this miniseries of exploring the monster taming mechanics of Final Fantasy XIII-2, it's time to start building the database and populating it with the data that we collected from the short script that we wrote <a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">in the last article</a>. The database will be part of a Ruby on Rails project, so we'll use the default SQLite3 development database. Before we can populate the database and start building the website around it, we need to make sure the data we parsed out of the FAQ is all okay with no typos or other corruption, meaning we need to validate our data. Once we do that, we can export it to a .csv file, start a new Rails project, and import the data into the database.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbF8tEPogrUxvw3X_wyCJUTm4oKOfGGUZmtydLy2gHy1L1xTOqdNFkj31t7nAnP2xQShl5fsPXDhUawwtyiacLC38Ha4L0YsYTIun2BIZcaRk1XSj6Rtjd8Wgnnj1JP8yzybZlyJlQVtw/s1600/Ravager_strike_abilities.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" data-original-height="450" data-original-width="800" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbF8tEPogrUxvw3X_wyCJUTm4oKOfGGUZmtydLy2gHy1L1xTOqdNFkj31t7nAnP2xQShl5fsPXDhUawwtyiacLC38Ha4L0YsYTIun2BIZcaRk1XSj6Rtjd8Wgnnj1JP8yzybZlyJlQVtw/s400/Ravager_strike_abilities.jpg" title="Final Fantasy XIII-2 battle scene" width="400" /></a></div>
<br />
<a name='more'></a><h4>
Validating a Collection of Data</h4>
Considering that we parsed out 164 monsters with dozens of properties each from the FAQ, we don't want to manually check all of that data to make sure every property that should be a number is a number and all property names are correctly spelled. That exercise would be way too tedious and error prone. This problem of validating the data sounds like it needs an extension to our script. Since we have the data in a list of hash tables, it should be fairly straightforward to create another hash table that can be used to validate each table in the list. The idea with this hash table is to have a set of valid properties as the keys in the table, and the values are regexes that should match each property value that they represent. These regexes will be more specific to each property, since those properties have already matched on the more general regexes that were used to collect the data in the first place. Additionally, every key in each monster hash should be in this template hash, and every template hash key should be in each monster hash. We could get even more detailed with our checks, but this validation should be enough to give us confidence in the data.<br />
<br />
To get started, we'll build up the first couple entries in the template hash and write the validation loop. Once it's working, we can fill out the rest of the entries more easily. Here are the name and minimum base HP entries along with the validation loop:
<script>hljs.initHighlightingOnLoad();</script>
<br />
<pre><code class="ruby">PROPER_NAME_REGEX = /^\w[\w\s]*\w$/
NUMBER_REGEX = /^\d+(:?,\d{3})?$/
VALID_MONSTER = {
"Name" => PROPER_NAME_REGEX,
"Minimum Base HP" => NUMBER_REGEX
}
data.each do |monster|
VALID_MONSTER.each do |key, regex|
if monster.key?(key)
unless monster[key] =~ regex
puts "Monster #{monster["Name"]} has invalid property #{key}: #{monster[key]}."
end
else
puts "Monster #{monster["Name"]} has missing property #{key}."
end
end
monster.each do |key, value|
unless VALID_MONSTER.key?(key)
puts "Monster #{monster["Name"]} has extra property #{key}: #{value}."
end
end
end</code></pre>
This is a fair amount of code, so let's take it in parts. First, we define two regexes for a proper name and a number. The proper name regex is the same as part of our previous property value regex in that it matches on multiple words separated by whitespace, but it has two extra symbols at the beginning and end. The '^' at the beginning means that the next character in the pattern has to appear at the start of the string, and the '$' at the end means that the last character that matches has to be at the end of the string. Together, these symbols mean that the entire string needs to match the regex pattern.<br />
<br />
The number regex is similar to the proper name regex, except that it matches on numbers instead of words. The <span style="font-family: "courier new" , "courier" , monospace;">(:?,\d{3})</span> group matches on a comma followed by three digits because the {3} pattern means that the previous character type, in this case a digit, must be repeated three times. This group is optional, so the regex will match on 1234 as well as 1,234. The number regex is also wrapped in a '^' and a '$' so that the entire string must match the pattern.<br />
<br />
The next constant is simply the start of our monster template hash with "Name" and "Minimum Base HP" entries. What follows is the validation loop, and it is laid out about how it was described. First, we iterate through each monster in the <span style="font-family: "courier new" , "courier" , monospace;">data</span> list that we have already populated with the monsters from the FAQ. Within each monster we iterate through every entry of the valid monster template. If the monster has the property we're looking at, we check if the property value matches the regex for that property. If it doesn't, we print out an error. If the property doesn't exist, we print out a different error. Then we iterate through every property of the monster, and if a property doesn't exist in the template, we print out another error.<br />
<br />
If we run this script now, we end up with a ton of errors for extra properties because we haven't added those properties to the template, yet. However, from looking at the first few monster's outputs, it appears that the other checks are working, so we can start filling out the rest of our template. We can quickly add in the obvious properties, checking the script periodically to make sure we haven't gone astray. The mostly finished template looks like this:<br />
<pre><code class="ruby">PROPER_NAME_REGEX = /^\w.*[\w)!%]$/
NUMBER_REGEX = /^\d+(:?,\d{3})?$/
SMALL_NUMBER_REGEX = /^(\d\d?\d?|N\/A)$/
PERCENTAGE_REGEX = /^(\d\d?\d?%|N\/A)$/
LIST_REGEX = /^((:?All )?\w+(:?, (:?All )?\w+)*|N\/A)$/
FREE_TEXT_REGEX = /^\S+(?:\s\S+)*$/
TIME_REGEX = /^\d\d?:\d\d$/
VALID_MONSTER = {
"Name" => PROPER_NAME_REGEX,
"Role" => PROPER_NAME_REGEX,
"Location" => PROPER_NAME_REGEX,
"Location2" => PROPER_NAME_REGEX,
"Location3" => PROPER_NAME_REGEX,
"Max Level" => SMALL_NUMBER_REGEX,
"Speed" => SMALL_NUMBER_REGEX,
"Tame Rate" => PERCENTAGE_REGEX,
"Minimum Base HP" => NUMBER_REGEX,
"Maximum Base HP" => NUMBER_REGEX,
"Minimum Base Strength" => SMALL_NUMBER_REGEX,
"Maximum Base Strength" => SMALL_NUMBER_REGEX,
"Minimum Base Magic" => SMALL_NUMBER_REGEX,
"Maximum Base Magic" => SMALL_NUMBER_REGEX,
"Growth" => PROPER_NAME_REGEX,
"Immune" => LIST_REGEX,
"Resistant" => LIST_REGEX,
"Halved" => LIST_REGEX,
"Weak" => LIST_REGEX,
"Constellation" => PROPER_NAME_REGEX,
"Feral Link" => PROPER_NAME_REGEX,
"Description" => FREE_TEXT_REGEX,
"Type" => PROPER_NAME_REGEX,
"Effect" => FREE_TEXT_REGEX,
"Damage Modifier" => FREE_TEXT_REGEX,
"Charge Time" => TIME_REGEX,
"PS3 Combo" => FREE_TEXT_REGEX,
"Xbox 360 Combo" => FREE_TEXT_REGEX,
"Default Passive1" => PROPER_NAME_REGEX,
"Default Passive2" => PROPER_NAME_REGEX,
"Default Passive3" => PROPER_NAME_REGEX,
"Default Passive4" => PROPER_NAME_REGEX,
"Default Skill1" => PROPER_NAME_REGEX,
"Default Skill2" => PROPER_NAME_REGEX,
"Default Skill3" => PROPER_NAME_REGEX,
"Default Skill4" => PROPER_NAME_REGEX,
"Default Skill5" => PROPER_NAME_REGEX,
"Default Skill6" => PROPER_NAME_REGEX,
"Default Skill7" => PROPER_NAME_REGEX,
"Default Skill8" => PROPER_NAME_REGEX,
"Special Notes" => FREE_TEXT_REGEX,
}</code></pre>
Notice that the <span style="font-family: "courier new" , "courier" , monospace;">PROPER_NAME_REGEX</span> pattern had to be relaxed to match on almost anything, as long as it starts with a letter and ends with a letter, ')', '!', or '%'. This compromise had to be made for skill names like "Strength +10%" or constellation names like "Flan (L)" or feral link names like "Items Please!" While these idiosyncrasies are annoying, the alternative is to make much more specific and complicated regexes. In most cases going to that extreme isn't worth it because the names that are being checked will be compared against names in other tables that we don't have, yet. Those data validation checks can be done later during data import when we have the other tables to check against. Waiting and comparing against other data reduces the risk that we introduce more errors from making the more complicated regexes, and we save time and effort as well.<br />
<br />
The location property has an odd feature that makes it a bit difficult to handle. Some monsters appear in up to three different areas in the game, but it's only a handful of monsters that do this. Having multiple locations combined in the same property is less than ideal because we'll likely want to look up monsters by location in the database, and we'll want to index that field so each location value should be a unique name, not a list. Additionally, the FAQ puts each location on a separate line, but not prefixed with the "Location-----:" property name. This format causes problems for our script. To solve both problems at once, we can add "Location2" and "Location3" properties anywhere that a monster has a second or third location by directly editing the FAQ.<br />
<br />
This template covers nearly all of the monster properties, except for the level skill and passive properties. We'll get to those properties in a second, but first we have another problem to fix. It turns out that the two location properties we added and the last three properties in the template don't always occur, so we have to modify our check on those properties slightly:<br />
<pre><code class="ruby">OPTIONAL_KEYS = [
"Location2",
"Location3",
"Default Passive1",
"Default Passive2",
"Default Passive3",
"Default Passive4",
"Default Skill1",
"Default Skill2",
"Default Skill3",
"Default Skill4",
"Default Skill5",
"Default Skill6",
"Default Skill7",
"Default Skill8",
"Special Notes"
]
# ...
elsif !OPTIONAL_KEYS.include? key
puts "Monster #{monster["Name"]} has missing property #{key}."
end
# ...</code></pre>
We simply change the <span style="font-family: "courier new" , "courier" , monospace;">else</span> branch of the loop that checks that all properties in the template are in the monster data so that it's an <span style="font-family: "courier new" , "courier" , monospace;">elsif</span> branch that only executes if the key is not one of those optional keys.<br />
<br />
Now we're ready to tackle the level properties. What we don't want to do here is list every single level from 1 to 99 for both skill and passive properties. There has to be a better way! The easiest thing to do is add a check for if the key matches the pattern of "Lv. XX (Skill|Passive)" in the loop that checks if each monster property exists in the template, and accept it if the key matches and the value matches the <span style="font-family: "courier new" , "courier" , monospace;">PROPER_NAME_REGEX</span>. This fix is shown in the following code:<br />
<pre><code class="ruby">LEVEL_PROP_REGEX = /^Lv\. \d\d (Skill|Passive)$/
# ...
monster.each do |key, value|
unless VALID_MONSTER.key?(key)
if key =~ LEVEL_PROP_REGEX
unless value =~ PROPER_NAME_REGEX
puts "Monster #{monster["Name"]} has invalid level property #{key}: #{value}."
end
else
puts "Monster #{monster["Name"]} has extra property #{key}: #{value}."
end
end
end
# ...</code></pre>
I tried to make the conditional logic as simple and self-explanatory as possible. I find that simpler is better when it comes to logic because it's easy to make mistakes and let erroneous edge cases through. If this logic was any more complicated, I would break it out into named functions to make the intent clearer still.<br />
<br />
With this addition to the data validation checks, we've significantly reduced the list of errors from the script output, and we can actually see some real typos that were in the FAQ. The most common typo was using "Lvl." instead of "Lv." and there are other assorted typos to deal with. We don't want to change the regexes to accept these typos because then they'll appear in the database, and we don't want to add code to the script to fix various random typos because that's just tedious nonsense. It's best to fix the typos in the FAQ and rerun the script. It's not too bad a task for these few mistakes.<br />
<br />
<h4>
Exporting Monsters to a CSV File</h4>
Now that we have this nice data set of all of the monster properties we could ever want, we need to write it out to a .csv file so that we can then import it into the database. This is going to be some super complicated code. Are you ready? Here it goes:<br />
<pre><code class="ruby">require 'csv'
opts = {headers: data.reduce(&:merge).keys, write_headers: true}
CSV.open("monsters.csv", "wb", opts) do |csv|
data.each { |hash| csv << hash }
end</code></pre>
Honestly, Ruby is one of my favorite languages. Things that you would think are complicated can be accomplished with ease. Because we already structured our data in a csv-friendly way as an array of hashes, all we have to do is run through each hash and write it out through the CSV::Writer with the '<span style="font-family: "courier new" , "courier" , monospace;"><<</span>' operator.<br />
<br />
We need to take care to enumerate all of the header names that we want in the .csv file, and that happens in the options that are passed to <span style="font-family: "courier new" , "courier" , monospace;">CSV.open</span>. Specifically, <span style="font-family: "courier new" , "courier" , monospace;">headers: data.reduce(&:merge).keys</span> tells the CSV::Writer what the list of header names is, and the writer is smart enough to put blank entries in wherever a particular header name is missing in the hash that it is currently writing out to the file. The way that code works to generate a list of header names is pretty slick, too. We simply tell the data array to use the <span style="font-family: "courier new" , "courier" , monospace;">Hash#merge</span> function to combine all of the hashes into one hash that contains all of the keys. Since we don't care about the values that got merged in the process, we simply grab the keys from this merged hash, and voila, we have our headers.<br />
<br />
The .csv file that's generated from this script is a real beast, with 204 unique columns for our 164 monsters. Most of those columns are the sparsely populated level-specific skills and passive abilities. We'll have to find ways to deal with this sparsely populated matrix when using the database, but it should be much better than dealing with one or two fields of long lists of abilities. At least, that's what I've read in books on database design. I'm learning here, so we'll see how this goes in practice.<br />
<br />
<h4>
Importing Monsters Into a Database</h4>
This part isn't going to be quite as easy as exporting because we'll need to write a database schema, but it shouldn't be too bad. Before we get to that, we need to create a new Ruby on Rails project. I'll assume Ruby 2.5.0 or higher and Rails 6.0 are installed. If not, see the start of this <a href="https://guides.rubyonrails.org/getting_started.html#creating-a-new-rails-project">Rails Getting Started guide</a> to get that set up. We start a new Rails project by going to the directory where we want to create it and using this Rails command:<br />
<pre><code class="bash">$ rails new ffxiii2_monster_taming</code></pre>
Rails generates the new project and a bunch of directories and files. Next, we descend into the new project and create a new model for monsters:<br />
<pre><code class="bash">$ cd ffxiii2_monster_taming
$ rails generate model Monster name:string</code></pre>
In Rails model names are singular, hence "Monster" instead of "Monsters." We also include the first database attribute that will be a part of the migration that is generated with this command. We could list out all 204 attributes in the command along with their data types, but that would be terribly tedious. There's an easier way to get them into the migration, which starts out with this code to create the Monster table:<br />
<pre><code class="ruby">class CreateMonsters < ActiveRecord::Migration[6.0]
def change
create_table :monsters do |t|
t.string :name
t.timestamps
end
end
end</code></pre>
All we have to do is add the other 203 attributes along with their data types and we'll have a complete table ready to generate, but how do we do this efficiently? Conveniently, we already have a list of the attribute names as the header line in the monsters.csv file. We just have to copy that line into another file and do some search-and-replace operations on it to get the list into a form that can be used as the code in this migration file.<br />
<br />
First, we'll want to make a couple changes in place so that the .csv header has the same names as the database attributes. This will make life easier when we import. All spaces should be replaced with underscores, and the periods in the "Lv." names should be removed. Finally, the whole line should be converted to lowercase to adhere to Rails conventions for attribute names. Once that's done, we can copy the header line to a new file, replace every comma with a newline character, and replace each beginning of a line with "<span style="font-family: "courier new" , "courier" , monospace;"> t.string </span>" to add in the attribute types. They are almost all going to be strings, and it's simple to go back and change the few that are not to integers, floats, and times. I did this all in Vim, but any decent text editor should be up to the task. Now we have a complete migration file:<br />
<pre><code class="ruby">class CreateMonsters < ActiveRecord::Migration[6.0]
def change
create_table :monsters do |t|
t.string :name
t.string :role
t.string :location
t.string :location2
t.string :location3
t.integer :max_level
t.integer :speed
t.string :tame_rate
t.string :growth
t.string :immune
t.string :resistant
t.string :halved
t.string :weak
t.string :constellation
t.integer :minimum_base_hp
t.integer :maximum_base_hp
t.integer :minimum_base_strength
t.integer :maximum_base_strength
t.integer :minimum_base_magic
t.integer :maximum_base_magic
t.string :feral_link
t.string :description
t.string :monster_type
t.string :effect
t.float :damage_modifier
t.time :charge_time
t.string :ps3_combo
t.string :xbox_360_combo
t.string :default_passive1
t.string :default_passive2
t.string :default_passive3
t.string :default_passive4
t.string :default_skill1
t.string :default_skill2
t.string :default_skill3
t.string :default_skill4
t.string :default_skill5
t.string :default_skill6
t.string :default_skill7
t.string :default_skill8
t.string :special_notes
t.string :lv_02_passive
t.string :lv_02_skill
#...
# over a hundred more lv_xx attributes
#...
t.string :lv_99_passive
t.string :lv_99_skill
t.timestamps
end
end
end</code></pre>
Now, we can run this migration with the command:<br />
<pre><code class="bash">$ rails db:migrate</code></pre>
And we have the beginnings of a monster table. We just need to populate it with our monsters. Rails 6.0 makes this task quite simple using a database seed file, and since we have the same names for the database attributes as the .csv file column headers, it's dead simple. In the lib/tasks/ directory, we can make a file called seed_monsters.rake with the following code:<br />
<pre><code class="ruby">require 'csv'
namespace :csv do
desc "Import Monster CSV Data"
task :import_monsters => :environment do
csv_file_path = 'db/monsters.csv'
CSV.foreach(csv_file_path, {headers: true}) do |row|
Model.create!(row.to_hash)
puts "#{row['name']} added!"
end
end
end</code></pre>
When we run this task, the code is going to loop through each line of the .csv file (that we make sure to put in db/monsters.csv), and create a monster in the database for each row in the file. We also print out the monster names so we can see it working. Then it's a simple matter of running this command:<br />
<pre><code class="bash">$ rails db:seed</code></pre>
And we see all of the monster names printed out to the terminal, and the database is seeded with our 164 monsters.<br />
<br />
We've accomplished a lot in this post with running some validation checks on the monster data, exporting it to a .csv file, creating a database table, and importing the monsters.csv file into that table. We still have plenty to do, creating and importing the other tables and relating the data between tables. That will be the goal for next time.<div><br /></div><div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div></div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-38370143151173013142020-02-17T21:07:00.001-06:002020-10-26T20:18:17.999-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2: Data CollectionThe monster taming aspect of Final Fantasy XIII-2 is surprisingly deep and complex, so much so that I'm interested in exploring it in this miniseries by shoving the monster taming data into a database and viewing and analyzing it with a website made in Ruby on Rails. <a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">In the last article</a>, we learned what monster taming is all about and what kind of data we would want in the database, basically roughing out the database design. Before we can populate the database and start building the website around it, we need to get that data into a form that's easy to import, so that's what we'll do today.<br />
<br />
<a name='more'></a><h4>
Starting a Data Parsing Script</h4>
<div>
We already identified a good source for most of the data we want to use from the <a href="https://gamefaqs.gamespot.com/ps3/619315-final-fantasy-xiii-2/faqs/63731">Monster Infusion FAQ</a> post on <a href="http://gamefaqs.com/">Gamefaqs.com</a>. However, we don't want to type the thousands of lines of data from this FAQ into our database because we would be introducing human error with the data copying, and the writers of this FAQ have already gone through all of the trouble of entering the data the first time, hopefully without mistakes. Besides, why would we go through such a tedious process when we could have fun writing a script to do the work for us? Come on, we're programmers! Let's write this script.</div>
<br />
Since the website will eventually be in Ruby on Rails, we might as well write this script in Ruby, too. It's not absolutely necessary to write the script in Ruby because it's a one-off deal that will only be run once (when it works) to convert the text file into a format that we can easily import into a database, but Ruby is pretty darn good at text processing, so let's stick with it. I like writing scripts in stages, breaking things down into simple problems and starting with an easy first step, so let's do that here. The simplest thing we can do is read in the text file after saving the FAQ to a local file. To add a bit of debug to make sure we have the file read in, let's scan through and print out the section header for the data we're looking for in the file:
<script>hljs.initHighlightingOnLoad();</script>
<br />
<pre><code class="ruby">File.foreach("ffiii2_monster_taming_faq.txt") do |line|
if line.include? "MLTameV"
puts line
end
end</code></pre>
Already, this code gives the basic structure of what we're trying to do. We're going to read in the file, loop through every line, look for certain patterns, and output what we find that matches those patterns. The real deal will be much more complex, but it's always good to have a working starting point.<br />
<br />
This code also has a few problems that we may or may not want to do anything about. First, it's just hanging out in the middle of nowhere. It's not in a class or function or anything more structured. If this was going to be a reusable parsing tool for converting various FAQs into rows of data, I would definitely want to engineer this code more robustly. But hey, this is a one-off script, and it doesn't <i>need</i> all of that extra support to make it reusable. Over engineering is just a waste of time so we'll leave this code out in the open.<br />
<br />
Second, I've got two constant strings hard-coded in those lines: the file name and the search string. I may want to stick the search string in a variable because it's not terribly obvious what "MLTameV" means. The file name, on the other hand, doesn't need to be in a variable. I plan to keep this part of the code quite simple, and it's the obvious loop where the file is read in. On top of that, this code will be very specific to handling this exact file, so I want the file name to be tightly coupled to this loop. If the script is ever copied and modified to work on a different file, this file name string can be changed in this one place to point to the new file that that script works with. I don't see a need to complicate this code with a variable.<br />
<br />
Third, when this code runs, it prints out two lines instead of one because there's another instance of "MLTameV" in the table of contents of the file. For locating the place to start parsing monster data, we want the second instance of this string. One way to accomplish this task is with the following code:<br />
<pre><code class="ruby">SECTION_TAG = "MLTameV"
section_tag_found = false
File.foreach("ffiii2_monster_taming_faq.txt") do |line|
if section_tag_found and line.include? SECTION_TAG
puts line
elsif line.include? SECTION_TAG
section_tag_found = true
end
end</code></pre>
Now only the section header line is printed when this script is run. However, as what inevitably happens when we add more code, we've introduced a new problem. It may not be obvious right now, but the path that we're on with the <span style="font-family: "courier new" , "courier" , monospace;">section_tag_found</span> variable is not sustainable. This variable is a piece of state that notifies the code when we've seen a particular pattern in the text file so we can do something different afterward. When parsing a text file using state variables like this one, we'll end up needing a <i>lot</i> of state variables, and it gets unmanageable and unreadable fast. What we are going to need instead, to keep track of what we need to do next, is a state machine.<br />
<br />
<h4>
Parsing Text with a Finite State Machine</h4>
<div>
Finite state machines (FSM) are great for keeping track of where you are in a process and knowing which state to go to next, like we need to know in the case of finding the section header for the list of tamable monsters in this text file. In the FSM we always have a current state that is one of a finite number of states, hence the name. Depending on the input in that state, the FSM will advance to a next state and possibly perform some output task. Here is what that process looks like in Ruby for finding the second section tag:</div>
<pre><code class="ruby">SECTION_TAG = "MLTameV"
section_tag_found = lambda do |line|
if line.include? SECTION_TAG
puts line
end
return section_tag_found
end
start = lambda do |line|
if line.include? SECTION_TAG
return section_tag_found
end
return start
end
next_state = start
File.foreach("ffiii2_monster_taming_faq.txt") do |line|
next_state = next_state.(line)
end</code></pre>
First, the states are defined as lambda methods so that they can easily be passed around as variables, but still called as functions. These variables have to be declared before they're used, so the <span style="font-family: "courier new" , "courier" , monospace;">section_tag_found</span> method either has to be defined first because the <span style="font-family: "courier new" , "courier" , monospace;">start</span> method uses it, or all methods could be predefined at the start of the file and then redefined with their method bodies in any desired order. Another way to define these states would be to wrap the whole thing in a class so that the states are class members, but that kind of design would be more warranted if this FSM was part of a larger system. As it is, this parser will be almost entirely made up of this FSM, so we don't need to complicate things.<br />
<br />
We can also represent this FSM with a diagram:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvAvQk_7ZbqIGw3BCgrexNOinYeOAJXjGuoXxncwa7mbCJWiIYFZotD4C6N-7FNO0LycQisWnyYJoNsTNBNyISHZCFpKwvk3iRdn74TZGT9oFw8ol5HNFXge8qWPf4icgvKhcg3lhb1Vk/s1600/fsm_tamable_monsters_start.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="242" data-original-width="572" height="168" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvAvQk_7ZbqIGw3BCgrexNOinYeOAJXjGuoXxncwa7mbCJWiIYFZotD4C6N-7FNO0LycQisWnyYJoNsTNBNyISHZCFpKwvk3iRdn74TZGT9oFw8ol5HNFXge8qWPf4icgvKhcg3lhb1Vk/s400/fsm_tamable_monsters_start.png" width="400" /></a></div>
<br />
The FSM starts in the Start state, obviously, and it transitions to the Section Tag Found state when there's a matching <span style="font-family: "courier new" , "courier" , monospace;">SECTION_TAG</span>. The unlabeled lines pointing back to the same states mean that for any other condition, the state remains unchanged. This diagram is quite simple, but when the FSM gets more complex, it will definitely help understanding to see it drawn out.<br />
<br />
Notice that running through the lines of the text file in the <span style="font-family: "courier new" , "courier" , monospace;">foreach</span> loop became super simple. All that's necessary is to feed each line into the <span style="font-family: "courier new" , "courier" , monospace;">next_state</span>, and assign the return value as the new <span style="font-family: "courier new" , "courier" , monospace;">next_state</span>. The current state is kind of hidden because we're assigning the <span style="font-family: "courier new" , "courier" , monospace;">next_state</span> to itself. Also notice that we need to be careful to always return a valid state in each path of each state method, even if it's the same state that we're currently in. Inadvertently returning something that was not a valid state would be bad, as the FSM is going to immediately try to call it on the next line.<br />
<br />
Now that we have an FSM started, it'll be easy to add more states and start working our way through the tamable monster data. What do we need to look for next? Well, we can take a look at the data for one monster and see if there are any defining characteristics:<br />
<pre><code class="text">...............................................................................
MONSTER 001
Name---------: Apkallu Minimum Base HP------: 1,877
Role---------: Commando Maximum Base HP------: 2,075
Location-----: Academia 500 AF Minimum Base Strength: 99
Max Level----: 45 Maximum Base Strength: 101
Speed--------: 75 Minimum Base Magic---: 60
Tame Rate----: 10% Maximum Base Magic---: 62
Growth-------: Standard
Immune-------: N/A
Resistant----: N/A
Halved-------: All Ailments
Weak---------: Fire, Lightning
Constellation: Sahagin
Feral Link-----: Abyssal Breath
Description----: Inflicts long-lasting status ailments on target and nearby
opponents.
Type-----------: Magic
Effect---------: 5 Hits, Deprotect, Deshell, Wound
Damage Modifier: 1.8
Charge Time----: 1:48
PS3 Combo------: Square
Xbox 360 Combo-: X
Default Passive: Attack: ATB Charge
Default Skill--: Attack
Default Skill--: Ruin
Default Skill--: Area Sweep
Lv. 05 Skill---: Powerchain
Lv. 12 Passive-: Strength +16%
Lv. 18 Skill---: Slow Chaser
Lv. 21 Skill---: Scourge
Lv. 27 Passive-: Strength +20%
Lv. 35 Passive-: Resist Dispel +10%
Lv. 41 Passive-: Strength +25%
Lv. 42 Passive-: Resist Dispel +44%
Lv. 45 Skill---: Ruinga
Special Notes: Apkallu only spawns twice in Academia 500 AF. If you fail to
acquire its Crystal in both encounters, you will have to close
the Time Gate and replay the area again.
...............................................................................</code></pre>
<div>
That series of dots at the beginning looks like a good thing to search for. It repeats at the start of every monster, so it's a good marker for going into a monster state. We'll also want to pass in a data structure that will be used to accumulate all of this monster data that we're going to find. To make it easy to export to a .csv file at the end, we're going to make this data structure an array of hashes, and it looks like this with the new state:</div>
<pre><code class="ruby">SECTION_TAG = "MLTameV"
MONSTER_SEPARATOR = "........................................"
new_monster = lambda do |line, data|
if line.include? MONSTER_SEPARATOR
return new_monster, data << {}
end
return new_monster, data
end
section_tag_found = lambda do |line, data|
if line.include? SECTION_TAG
return new_monster, data
end
return section_tag_found, data
end
start = lambda do |line, data|
if line.include? SECTION_TAG
return section_tag_found, data
end
return start, data
end
next_state = start
data = []
File.foreach("ffiii2_monster_taming_faq.txt") do |line|
next_state, data = next_state.(line, data)
end
puts data.length</code></pre>
I shortened the <span style="font-family: "courier new" , "courier" , monospace;">MONSTER_SEPARATOR</span> pattern in case there were some separators that were shorter than the first one, but it should still be plenty long to catch all of the instances of separators between monsters in the file. Notice that we now have to pass the <span style="font-family: "courier new" , "courier" , monospace;">data</span> array into and out of each state method so that we can accumulate the monster data in it. Right now it simply appends an empty hash for each monster it finds. We'll add to those hashes in a bit. At the end of the script, I print out the number of monsters found, which we expect to be 164, and it turns out to be a whopping 359! That's because that same separator is used more after the tamable monster section of the file, and we didn't stop at the end of the section. That should be easy enough to fix:<br />
<pre><code class="ruby">SECTION_TAG = "MLTameV"
MONSTER_SEPARATOR = "........................................"
NEXT_SECTION_TAG = "SpecMon"
end_monsters = lambda do |line, data|
return end_monsters, data
end
new_monster = lambda do |line, data|
if line.include? MONSTER_SEPARATOR
return new_monster, data << {}
elsif line.include? NEXT_SECTION_TAG
return end_monsters, data
end
return new_monster, data
end
# ...</code></pre>
I added another state <span style="font-family: "courier new" , "courier" , monospace;">end_monsters</span> that consumes every line to the end of the file, and we enter that state from the <span style="font-family: "courier new" , "courier" , monospace;">new_monster</span> state if we see the <span style="font-family: "courier new" , "courier" , monospace;">NEXT_SECTION_TAG</span>. Now if we run the script again, we get a count of 166 monsters. Close, but still not right. The problem is that there are a couple extra separator lines used in the tamable monster section, one after the last monster and one extra separator after a sub-heading for DLC monsters. We're going to have to get a bit more creative with how we detect a new monster. If we look back at the example of the first monster, we see that after the separator the next text is MONSTER 001. This title for each monster is consistent for all of the monsters, with MONSTER followed by a three digit number. Even the DLC monsters have this tag with DLC in front of it. This pattern is perfect for matching on a regular expression (regex).<br />
<br />
<h4>
Finding Monster Data with Regular Expressions</h4>
<div>
A regex is a text pattern defined with special symbols that mean various things like "this character is repeated one or more times" or "any of these characters" or "this character is a digit." This pattern can be used to search a string of text, which is called matching the regex. In Ruby a regex pattern is denoted by wrapping it in forward slashes (/), and we can easily define a regex for our MONSTER 001 pattern:</div>
<pre><code class="ruby">SECTION_TAG = "MLTameV"
MONSTER_SEPARATOR = "........................................"
NEXT_SECTION_TAG = "SpecMon"
NEW_MONSTER_REGEX = /MONSTER\s\d{3}/
find_separator = nil
end_monsters = lambda do |line, data|
return end_monsters, data
end
new_monster = lambda do |line, data|
if NEW_MONSTER_REGEX =~ line
return find_separator, data << {}
elsif line.include? NEXT_SECTION_TAG
return end_monsters, data
end
return new_monster, data
end
find_separator = lambda do |line, data|
if line.include? MONSTER_SEPARATOR
return new_monster, data
end
return find_separator, data
end
# ...</code></pre>
The <span style="font-family: "courier new" , "courier" , monospace;">NEW_MONSTER_REGEX</span> is defined as the characters MONSTER, followed by a space (\s), followed by three digits (\d). I changed the new_monster state to look for a match on our new regex, and added a <span style="font-family: "courier new" , "courier" , monospace;">find_separator</span> state to still search for the <span style="font-family: "courier new" , "courier" , monospace;">MONSTER_SEPARATOR</span>. Notice that the FSM will bounce between these two states, so the state that's defined later has to be declared at the top of the file, otherwise Ruby will complain that <span style="font-family: "courier new" , "courier" , monospace;">find_separator</span> is undefined in <span style="font-family: "courier new" , "courier" , monospace;">new_monster</span>.<br />
<br />
These regex patterns are useful and powerful, but they can also be quite tricky to get right, especially when they get long and complicated. We'll be using them to pull out all of the data we want from each monster, but we'll try to keep them as simple as possible. The next regex is more complicated, but it will allow us to pull nearly all of the properties for each monster and put it into the empty hash that was added to the list of hashes for that monster. Ready? Here it is:<br />
<pre><code class="ruby">MONSTER_PROP_REGEX = /(\w[\w\s\.]*\w)-*:\s(\S+(?:\s\S+)*)/</code></pre>
We'll break this regex apart and figure out what each piece means separately.<br />
<br />
The first part of the regex, <span style="font-family: "courier new" , "courier" , monospace;">(\w[\w\s\.]*\w)</span>, is surrounded by parentheses and is called a capture. A capture will match on whatever the pattern is inside the parentheses and save that matching text so that it can be accessed later. We'll see how that works in the code a little later, but right now we just need to know that this is how we're going to separate out the property name and its value from the full matching text. This particular capture is the property name, and it starts with a letter or number, symbolized with \w. The stuff in the brackets means that the next character can be a letter or number, a space, or a period. Any of those characters will match. Then the following '*' means that a string of zero or more of the preceding character will match. Finally, the property name must end with a letter or number, symbolized with \w again. The reason this pattern can't just be a string of letters and numbers is because some of the property names are multiple words, and the "Lv. 05 Skill" type properties also have periods in them. We want to match on all of those possibilities.<br />
<br />
The next part of the regex is <span style="font-family: "courier new" , "courier" , monospace;">-*:\s</span>, which simply means it will match on zero or more '-', followed by a ':', followed by a space. Reviewing the different lines for the MONSTER 001 example above, we can see that this pattern is indeed what happens. Some cases have multiple dashes after the property name, while others are immediately followed by a colon. The colon is always immediately followed by a single space, so this should work well as our name-value separator. It's also outside of any parentheses because we don't want to save it for later.<br />
<br />
The last part of the regex is another capture for the property value: <span style="font-family: "courier new" , "courier" , monospace;">(\S+(?:\s\S+)*)</span>. The \S+—note the capital S—will match on one or more characters that are not white space. It's the inverse of \s. The next thing in this regex looks like yet another capture, but it has this special '?:' after the open parenthesis. This special pattern is called a grouping. It allows us to put a repeat pattern after the grouping, like the '*' in this case, so that it will match on zero or more of the entire grouping. It will not save it for later, though. Since this grouping is a space followed by one or more non-space characters, this pattern will match on zero or more words, including special characters. If we look at the example monster above, we see that this pattern is exactly what we want for most of the property values. Special characters are strewn throughout, and it would be too much trouble to enumerate them all without risking missing some so we cover our bases this way.<br />
<br />
Fairly simple, really. We're going to match on a property name made up of one or more words, followed by a dash-colon separator, and ending with a property value made up of one or more words potentially including a mess of special characters. Note how we couldn't have used the \S character for the property name because it would have also matched on and consumed the dash-colon separator. We also could not have used the [\s\S]* style pattern for the words in the property value because it would have matched on any number of spaces between words. That wouldn't work for the first few lines of the monster properties because there are two name-value pairs on those lines. Now that we have our regex, how do we use those captured names and values, and how exactly is this going to work for the lines with two pairs of properties on them? Here's what the new <span style="font-family: "courier new" , "courier" , monospace;">add_property</span> state looks like with some additional context:<br />
<pre><code class="ruby"># ...
MONSTER_PROP_REGEX = /(\w[\w\s\.]*\w)-*:\s(\S+(?:\s\S+)*)/
find_separator = nil
new_monster = nil
end_monsters = lambda do |line, data|
return end_monsters, data
end
add_property = lambda do |line, data|
props = line.scan(MONSTER_PROP_REGEX)
props.each { |prop| data.last[prop[0]] = prop[1] }
return new_monster, data if line.include? MONSTER_SEPARATOR
return add_property, data
end
new_monster = lambda do |line, data|
if NEW_MONSTER_REGEX =~ line
return add_property, data << {}
elsif line.include? NEXT_SECTION_TAG
return end_monsters, data
end
return new_monster, data
end
# ...</code></pre>
The double-property lines are handled with a different type of regex matcher, <span style="font-family: "courier new" , "courier" , monospace;">line.scan(MONSTER_PROP_REGEX)</span>. This scan returns an array of all of the substrings that matched the given regex in the string that it was called on. Conveniently, if the regex contains captures, the array elements are themselves arrays of each of the captures. For example, the scan of the first property line of our MONSTER 001 results in this array:<br />
<pre><code class="ruby">[['Name', 'Apkallu'],['Minimum Base HP', '1,877']]</code></pre>
We can simply loop through this array, adding property name and property value to the last hash in the list of hashes. Then, if the line was actually the MONSTER_SEPARATOR string, it didn't match any properties and we'll move on to the next monster. Otherwise, we stay in the <span style="font-family: "courier new" , "courier" , monospace;">add_property</span> state for the next line.<br />
<br />
This process works really well until we get to the "Default Passive" and "Default Skill" properties, because there can be multiple instances of those. In this case, we need to append a number to each of these properties, such as "Default Passive1", "Default Passive2", etc., to differentiate them so later instances don't overwrite earlier instances of each property. We can do this by modifying the props.each line to check for these default properties and append an incrementing number to their names:<br />
<pre><code class="ruby"> props.each do |prop|
if prop[0] == 'Default Passive' || prop[0] == 'Default Skill'
n = 1
n += 1 while data.last.has_key? (prop[0] + n.to_s)
prop[0] += n.to_s
end
data.last[prop[0]] = prop[1]
end</code></pre>
This fix takes care of multiple instances of the same property, but one last thing that we're not handling is those multi-line descriptions and special notes. We need to append those lines to the correct property when we come across them, but how do we do that? Keep in mind that these extra lines won't match on MONSTER_PROP_REGEX, so we can simply detect that non-match, make sure it's not an empty line, and add it to the special notes if it exists or the description if the special notes doesn't exist. Here's what that code looks like in <span style="font-family: "courier new" , "courier" , monospace;">add_property</span>.<br />
<pre><code class="ruby">MONSTER_PROP_EXT_REGEX = /\S+(?:\s\S+)*/
# ...
add_property = lambda do |line, data|
props = line.scan(MONSTER_PROP_REGEX)
props.each do |prop|
if prop[0] == 'Default Passive' || prop[0] == 'Default Skill'
n = 1
n += 1 while data.last.has_key? (prop[0] + n.to_s)
prop[0] += n.to_s
end
data.last[prop[0]] = prop[1]
end
return new_monster, data if line.include? MONSTER_SEPARATOR
ext_line_match = MONSTER_PROP_EXT_REGEX.match(line)
if props.empty? and ext_line_match
if data.last.key? 'Special Notes'
data.last['Special Notes'] += ' ' + ext_line_match[0]
else
data.last['Description'] += ' ' + ext_line_match[0]
end
end
return add_property, data
end</code></pre>
By putting the extra code after the return if the line is the MONSTER_SEPARATOR, we can assume that this line is not the MONSTER_SEPARATOR and just check if the MONSTER_PROP_REGEX didn't match and there's <i>something</i> on the line. Then decide on which property to add the line to, and we're good to go.<br />
<br />
Okay, that was a lot of stuff, so let's review. First, we read in the file that we wanted to parse that contains most of the monster taming data we need. Then, we loop through the lines of the file, feeding them into a FSM in order to find the section of the file where the list of monsters is and separate each monster's properties into its own group. Finally, we use a few simple regex patterns to capture each monster's property name-value pairs and add them to a list of hashes that will be fairly easy to print out to a .csv file later. All of this was done in 66 lines of Ruby code! Here's the program in full so we can see how it all fits together:<br />
<pre><code class="ruby">SECTION_TAG = "MLTameV"
MONSTER_SEPARATOR = "........................................"
NEXT_SECTION_TAG = "SpecMon"
NEW_MONSTER_REGEX = /MONSTER\s\d{3}/
MONSTER_PROP_REGEX = /(\w[\w\s\.]*\w)-*:\s(\S+(?:\s\S+)*)/
MONSTER_PROP_EXT_REGEX = /\S+(?:\s\S+)*/
find_separator = nil
new_monster = nil
end_monsters = lambda do |line, data|
return end_monsters, data
end
add_property = lambda do |line, data|
props = line.scan(MONSTER_PROP_REGEX)
props.each do |prop|
if prop[0] == 'Default Passive' || prop[0] == 'Default Skill'
n = 1
n += 1 while data.last.has_key? (prop[0] + n.to_s)
prop[0] += n.to_s
end
data.last[prop[0]] = prop[1]
end
return new_monster, data if line.include? MONSTER_SEPARATOR
ext_line_match = MONSTER_PROP_EXT_REGEX.match(line)
if props.empty? and ext_line_match
if data.last.key? 'Special Notes'
data.last['Special Notes'] += ' ' + ext_line_match[0]
else
data.last['Description'] += ' ' + ext_line_match[0]
end
end
return add_property, data
end
new_monster = lambda do |line, data|
if NEW_MONSTER_REGEX =~ line
return add_property, data << {}
elsif line.include? NEXT_SECTION_TAG
return end_monsters, data
end
return new_monster, data
end
find_separator = lambda do |line, data|
if line.include? MONSTER_SEPARATOR
return new_monster, data
end
return find_separator, data
end
section_tag_found = lambda do |line, data|
if line.include? SECTION_TAG
return find_separator, data
end
return section_tag_found, data
end
start = lambda do |line, data|
if line.include? SECTION_TAG
return section_tag_found, data
end
return start, data
end
next_state = start
data = []
File.foreach("ffiii2_monster_taming_faq.txt") do |line|
next_state, data = next_state.(line, data)
end</code></pre>
And here's the corresponding FSM diagram:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYmuKcVJWkSyyZVy4OBiahOQU6bPW8nG86QLOV7SDH1ehoTp371xlPNudU4vRcfDuOoufes-6qhJYMT7wM50R6IGl4eUEF2jFP43JDRUOU9Hk8Y_uf_2EtxakiQnByrvJeq2X3MLlgVvQ/s1600/fsm_tamable_monster_final.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Final FSM diagram of tamable monster parser" border="0" data-original-height="545" data-original-width="951" height="364" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYmuKcVJWkSyyZVy4OBiahOQU6bPW8nG86QLOV7SDH1ehoTp371xlPNudU4vRcfDuOoufes-6qhJYMT7wM50R6IGl4eUEF2jFP43JDRUOU9Hk8Y_uf_2EtxakiQnByrvJeq2X3MLlgVvQ/s640/fsm_tamable_monster_final.png" title="" width="640" /></a></div>
<br />
We still need to write the collected data out to a .csv file so that we can import it into a database, but that is a task for next time. Also, notice that we have done almost no data integrity checks on this input other than what the FSM and regex patterns inherently provide. Any mistakes, typos, or unexpected text in the file will likely result in missing or corrupt data, so we'll need to do some checks on the data as well. Additionally, this data is just the tamable monster data. We still need the other table data for abilities, game areas, monster materials, and monster characteristics. However, this is a great start on the data that was the most difficult to get, and we ended up with quite a few extra properties that we weren't intending to collect in the list. That's okay, I'm sure we'll find a use for them.<div><br /></div><div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div></div>Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-27710669887762638132020-01-27T21:18:00.001-06:002020-10-26T20:17:56.261-05:00Exploring Monster Taming Mechanics in Final Fantasy XIII-2Let's just get this out of the way. I'm a huge Final Fantasy fan. I adore the original game even today, Final Fantasy VI is definitely the best of the franchise, and I've found things to enjoy in every one that I've played, which is nearly all of the main-line games. (I still haven't managed to crack open FFIII, but I plan to soon.) Even each of the games in the Final Fantasy XIII trilogy had something that drew me in and kept me going through the game, wanting to learn more. These games get a lot of flack for being sub-par installments in the Final Fantasy franchise, and some of the criticism is warranted. The story is convoluted, and the plot is as confusing as quantum mechanics.<br />
<br />
That debate, however, is not why we're here now. We're here to look at one of the great aspects of FFXIII-2: the monster taming and infusion system.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijuF07R2LtZN06_OAlfBIaueIU1919Epy25dAGTx14W6ktwKr2EaksPFC0s0-oNFa_QBgfpQ32OkHhlg5kl0wAODLhJLdxDLFcYcpK-R_huotel8mwxpwKklZG-LOhSvN15TpD6nh5p1U/s1600/Paradigm_Pack_menu.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="720" data-original-width="1280" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijuF07R2LtZN06_OAlfBIaueIU1919Epy25dAGTx14W6ktwKr2EaksPFC0s0-oNFa_QBgfpQ32OkHhlg5kl0wAODLhJLdxDLFcYcpK-R_huotel8mwxpwKklZG-LOhSvN15TpD6nh5p1U/s400/Paradigm_Pack_menu.jpg" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<a name='more'></a>This system is deep and complex, but not in the off-putting way that the plot is. The amount of variety and configurability in the monsters that you can recruit to fight alongside Serah and Noel is astonishing, which is nice because those two are somewhat lacking in that department. Their development paths are fairly linear. There are a few choices about what strengths to give them as they level up in the "crystarium," but it's mostly a matter of ordering the abilities they learn and doesn't make much difference in the end. The monsters, on the other hand, allow for huge variations in development that results in a system with fascinating choices for optimization and prioritization. Figuring out how to capture and develop powerful monsters early in the game is great fun, and its this characteristic of Final Fantasy games—of finding ways to build a strong party early in the game without tedious grinding—that I really enjoy.<br />
<br />
On a totally different note, I've been considering different ways to practice using databases and building simple websites, and the monster taming system is complex enough and interesting enough that it would make for a good collection of content to use for that practice while having some fun in the process. So, the other goal of this series, other than exploring the monster taming system in FFXIII-2, is to explore how to get a data set into a database, put it up on a website with Ruby on Rails, and experiment with that data set in some novel ways. Before we get into all of that, however, we need to understand what the heck we're putting in the database in the first place, and to do that we need to understand this monster taming system in detail.<br />
<br />
<h4>
Monster Taming</h4>
<div>
Okay, what is this monster taming, and why is it so complicated? We'll have to start at the beginning and work our way through the system. At the start of the game there's just Serah, trying to stay alive. (Actually, the very start of the game is a flashy battle sequence and confusing plot points with Lightning, but let's ignore that.) Pretty soon Noel shows up and decides to help Serah out, so now it's a party of two. This setup goes on for a couple levels, but it's pretty weird for a Final Fantasy game. Normally there's three or four characters in a party. Then we come to a pitched battle with a Cait Sith and a Zwerg Scandroid. There will be a lot of weird monster names throughout this series. You're just going to have to roll with it. Anyway, after creaming the cat and the droid, they turn into monster crystals, which are basically the essence of monsters. These crystals are stored in your monster inventory, and you can assign three of them to coveted spots in your "Paradigm Pack" (not the name I would have picked). These three monster spirits will fight with you in battle, and so begins your long and precarious journey as the monster whisperer.</div>
<div>
<br /></div>
<div>
Monsters come in six basic varieties, conveniently matching the six roles that Serah and Noel can assume. These six role are briefly explained as follows:</div>
<div>
<ul>
<li>Commando - The Arnold Schwarzenegger role, plenty of strength, short on finesse.</li>
<li>Ravager - The mage role, uses attack magic.</li>
<li>Sentinel - This is a tank role, not many attacking options, but it absorbs damage like it's nothing.</li>
<li>Saboteur - Weakens the enemy by removing protections and inflicting ailments like poison and slow.</li>
<li>Synergist - Strengthens the party with offensive and defensive enhancements. Also moonlights as a business executive.</li>
<li>Medic - Heals the party, and possibly the only obvious role name of the bunch.</li>
</ul>
<div>
While Serah and Noel can switch between these roles, the tamed monsters have fixed roles. Switching the monster's role switches the monster, and there are three role possibilities for any given battle corresponding to the three monsters that are on deck. The monster's type is only the beginning of what a monster is, though. There is so much more.</div>
</div>
<div>
<br />
<h4>
Monster Training</h4>
</div>
<div>
Monsters can gain experience just like Serah and Noel. Both the humans and the monsters have crystariums where they advance along a path to gain abilities and increase their stats. While the humans have a crystarium for each of their six roles, the monsters each have one crystarium, possibly with multiple levels, where they gain their abilities and stats. Because the crystariums of the monsters are more unique to the monster itself, each monster type will learn a unique set of abilities and end up with different stats. Additionally, while the humans can move through their crystariums by spending crystarium points gained from winning battles, monsters can only advance on their crystariums by using various types of monster materials that are dropped by defeating monsters in battle. This seems to be some form of cannibalism, but it's pretty mild because the materials are bolts and orbs and other things like that. Monsters require different materials for their crystariums depending on what level they're on their crystarium, and if they're biological or mechanical monsters. Different materials will also give different bonuses to the monster's health, strength, magic, or all three stats.<br />
<br />
Following so far? Because we're just getting started. This monster whispering is intricate stuff. On top of the unique upgrade paths, abilities, and materials, each monster spirit has a set of characteristics that relate to how they will develop as they level up. A monster can be a "late bloomer," meaning it may be weak to start with, but it can reach the upper levels 70-99 of its crystarium. Maybe the monster is "brainy," meaning that it will learn lots of abilities, or it's "flameproof," which is pretty self-explanatory. There are 29 characteristics in all, and any given monster can have up to four of them. Monsters will also come with some initial abilities, whether that be actions like casting certain spells and attacking or passive abilities like "armor breaker" that allows it to penetrate an enemy's physical defense. Taming and training monsters are not the only ways to get monsters with certain abilities, however. This is where things get real, as in real complicated.<br />
<br />
<h4>
Monster Infusion</h4>
</div>
<div>
The third way to give a particular monster new and wonderful abilities is to take another monster that has the desired ability(ies) and fuse them into the desired monster through a magical monster infusion process. How does this work exactly? Who knows! How did materia work in FFVII, or guardian force cards in FFVIII? It's a Final Fantasy game; some things you just have to accept without question and move on. The source monster spirit is lost in this process of infusing the target monster with new abilities. It's a destructive process.</div>
<div>
<br /></div>
<div>
Losing the source monster is not the only cost, though. There are restrictions as well. The first restriction is that a monster can only have 10 passive abilities. If a monster accumulates more than 10 passive abilities, some of them are going to have to go. These abilities all have a rank, and higher ranked abilities will stick to a monster better than lower ranked abilities. Also, newer abilities are stickier than older abilities, according to when the monster learned them. The lowest ranked abilities will get the boot first, with order of acquisition being the tie-breaker—first in, first out.</div>
<div>
<br /></div>
<div>
The next restriction is red-locked abilities. These are abilities that cannot be transferred to or removed from a monster, ever. This restriction is pretty simple, unlike the next one.</div>
<div>
<br /></div>
<div>
Monsters can also have yellow-locked abilities, although these locks never exist by default. Yellow locks can be created, propagated, and destroyed by infusing abilities of the same type in various ways. Two abilities are the same type if they modify the same attribute. For example, HP +10% and HP +25% would be of the same type. Also, HP +10% is a lower rank than HP +25%. That's important for yellow locks because if you infuse a monster that already has a lower rank ability with an equal or higher rank ability of the same type, the infused ability comes with a yellow lock and will stay put when the monster's abilities overflow. Generally, if an ability of higher rank is added to a yellow-locked ability of lower rank, the yellow lock is kept. If an ability of equal or lower rank is added to a yellow-locked ability, the yellow lock is destroyed. It's a bit more complicated than that because there are about a dozen different combinations, but this summary should be sufficient for the purpose of setting the requirements of the database. Basically, we want to make sure we know the rank of each ability so that we can figure out the best way to develop monsters' abilities.<br />
<br />
All of the red lock and yellow lock stuff has to do with passive abilities, but there are two other types of abilities that come into play with monster infusion: role abilities and hidden abilities. Role abilities are the actions that the monster will take in battle, and there is no limit to the number of these abilities that a monster can have. When a source monster is infused, you can choose from its role abilities up to the number that its crystarium stage is at, which will be 1-5 depending on how much you can level up the monster and how much you actually leveled it up. The disadvantage of infusing too many role abilities on a monster is that you don't have control over what it does in battle, and if it has too many options, it probably won't be doing what you want it to do when you need it most. Decide what you're going to use a monster for, and then don't give it choices. You can't remove role abilities once they're infused.<br />
<br />
Lastly, hidden abilities are learned by a monster when it is infused with 99 levels worth of monsters of the opposite role. Commando and Ravager are opposites (makes sense), Saboteur and Synergist are opposites (makes even more sense), and Sentinel and Medic are opposites (the leftovers, I guess). For example, you could infuse nine level 17 Zwerg Scandroids onto your Red Chocobo, and it'll learn Jeopardize, which boosts the bonus the chocobo gets when attacking a staggered enemy. Each role has it's own hidden ability that it gets when those 99 levels of monsters of the opposite role are infused into it.<br />
<br />
<h4>
Acquiring the Data for the Database</h4>
</div>
<div>
Okay, that was a bunch of intricate, complicated stuff, but it gives us a good idea of what kind of data we want to put in our database so we can link it up and ask interesting questions about monster infusion. </div>
<div>
<br /></div>
<div>
First, we want to know all about monsters:</div>
<div>
<ul>
<li>What's its name?</li>
<li>Is it tamable? We might have a separate tamable monster table since most of the following properties wouldn't apply to non-tamable monsters.</li>
<li>What materials does it drop in battle?</li>
<li>What's its role?</li>
<li>Where in the game can we find it?</li>
<li>What are its characteristics?</li>
<li>What's its max level?</li>
<li>What are its starting and ending stats (HP, strength, and magic)?</li>
<li>What are its starting abilities?</li>
<li>What abilities does it learn and at which crystarium levels?</li>
<li>How many crystarium stages does it have?</li>
<li>What is its feral link? (We didn't talk about this. It's a special action that can be triggered when the monster gets hit too much.)</li>
<li>What does the feral link do?</li>
<li>We could also include pictures if we want to get fancy.</li>
</ul>
<div>
We also want to know about abilities. This will be a separate table:</div>
</div>
<div>
<ul>
<li>What's its name?</li>
<li>What's its rank?</li>
<li>What does it do?</li>
<li>Is it passive, role, or hidden? These may be separate tables, since they're different enough to warrant it.</li>
<li>Which role is it associated with?</li>
</ul>
<div>
We'll be interested in at least one aspect of the areas in the game:</div>
<div>
<ul>
<li>What's its name?</li>
<li>How early can we reach this area? I.e. which area unlocks this area?</li>
</ul>
</div>
<div>
Since there's a fair number of monster materials, we'll want to keep track of those:</div>
<div>
<ul>
<li>What's its name?</li>
<li>Is it biological or mechanical?</li>
<li>What stage of the crystarium is it for?</li>
<li>Does it boost HP, strength, magic, or all three? (The name does give this away, but let's be thorough.)</li>
</ul>
</div>
<div>
We'll also want to know a little about the monster characteristics because the names are not self-explanatory:</div>
</div>
<div>
<ul>
<li>What's its name?</li>
<li>What does it mean?</li>
</ul>
<div>
This is shaping up to be a reasonably complex database with 5-8 tables interlinked by these different items' names. The relations in the database will happen through IDs, but everything does have a name as well. The names will be what appears in the tables presented as views of the database, likely with hyperlinks to their information in their own tables. So how should we get all of this data into a database? I certainly don't want to enter it by hand. There's over 150 monsters, dozens of abilities, and dozens of properties for each monster. </div>
<div>
<br /></div>
<div>
Luckily, some ambitious people have already done the hard work of writing out all of these things in an FAQ, and it's available on <a href="http://gamefaqs.com/">gamefaqs.com</a>. The <a href="https://gamefaqs.gamespot.com/ps3/619315-final-fantasy-xiii-2/faqs/63731">Monster Infusion FAQ</a> by sakurayule, BMSirius, and Taly932 contains almost everything we want to put in the database. It also contains example builds for post-game super monsters, but we're going to look at something a bit different with this series. We want to figure out the best monster builds we can do during the game in order to have monsters that can help us through the game without the need to do any grinding. All of the necessary information is in that FAQ. We just have to write a script to parse it and put it in a form that's easy to import into the database. That parsing script is what we'll figure out how to write in the next episode.</div><div><br /></div><div><div><br /></div><div><b>Exploring Monster Taming Mechanics Table of Contents:</b></div><div><a href="https://sam-koblenski.blogspot.com/2020/01/exploring-monster-taming-mechanics-in.html">Part 1: Introduction</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/02/exploring-monster-taming-mechanics-in.html">Part 2: Data Collection</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in.html">Part 3: Data Validation and Database Import</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/03/exploring-monster-taming-mechanics-in_30.html">Part 4: Relational Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/04/exploring-monster-taming-mechanics-in.html">Part 5: The Remaining Tables</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/05/exploring-monster-taming-mechanics-in.html">Part 6: Viewing Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in_16.html">Part 7: Viewing More Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/06/exploring-monster-taming-mechanics-in.html">Part 8: Viewing the Monster Data</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/07/exploring-monster-taming-mechanics-in.html">Part 9: Viewing More Monster Data and Abilities</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in.html">Part 10: Filtering Monsters</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/08/exploring-monster-taming-mechanics-in_24.html">Part 11: Asking Deeper Questions</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/09/exploring-monster-taming-mechanics-in.html">Part 12: Finding Monster Materials</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in.html">Part 13: Monster Characteristics</a></div><div><a href="https://sam-koblenski.blogspot.com/2020/10/exploring-monster-taming-mechanics-in_26.html">Part 14: Dogfooding</a></div></div>
</div>
Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-50683246726870496852020-01-06T20:12:00.000-06:002020-01-06T20:12:25.264-06:00The Year in Review, Just the Leisure TimeLast January I did a review of how I spent my leisure time the previous year, and I set down a few expectations for the coming year, now past. It's time to look back and see how my actual activities stacked up to my expectations, and maybe learn something for the fresh year to come in 2020. I had big ambitions between reading, blogging, and playing, and not all of them were achieved. But, that's okay because it makes it easier to figure out what I want to do this year—some of what I didn't finish last year, and some new ideas and desires. How I spend my leisure time is very important to me. It should be at the same time relaxing and reinvigorating, enjoyable and enriching, soothing and stimulating. If one thing is obvious, it's that I still love to read because it hits all of those notes, and that is likely to continue in the year(s) to come.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgL-z6oefoZMfjIlC7Y7hqFtG9RFn1z8hvMI5V_jn6_y3nD97O-DJC0wxYWlYSREmlT7-sja6uPgnv8CfK42v1p71dAa_ULq8ETIJveYV6rc3wls52TctiQesoipWo4hatTQGoP8KnB7pA/s1600/love-books.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="534" data-original-width="800" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgL-z6oefoZMfjIlC7Y7hqFtG9RFn1z8hvMI5V_jn6_y3nD97O-DJC0wxYWlYSREmlT7-sja6uPgnv8CfK42v1p71dAa_ULq8ETIJveYV6rc3wls52TctiQesoipWo4hatTQGoP8KnB7pA/s400/love-books.jpg" width="400" /></a></div>
<br />
<a name='more'></a><br />
<h4>
Blog Posts</h4>
<div>
I'll start out again with what I've done on my blog, and I very nearly did everything I set out to do. I wrote 17 posts (not counting this one) instead of the predicted 18 because of how the 3-week intervals fell. They were almost all Tech Book Face Offs, except for the last two that were a <a href="https://sam-koblenski.blogspot.com/2019/11/tech-books-i-will-read-again.html">general review of books I will read again</a> and <a href="https://sam-koblenski.blogspot.com/2019/12/sharpen-your-programming-tools.html">programming practice sites that I've enjoyed</a>, and a random <a href="https://sam-koblenski.blogspot.com/2019/04/physics-book-face-off-hidden-reality-vs.html">Physics Book Face Off</a> that I threw in there in April to mix things up.</div>
<div>
<br /></div>
<div>
The most popular post by far was the review of books that I’ll read again, which racked up over twice as many hits as the next most popular post, and it was pretty fun to write as well. This was not your normal list of best books because I'm tired of reading best-programming-book lists that put up the same old safe books like CLRS or Knuth's <i>The Art of Computer Programming</i>. This post had a little of that just because some of those books are great to reread, but these were all genuinely books that I want to read again, not just because they're instructive, but because they're enjoyable reads. They make my brain tingle. They get me excited about programming and learning (or relearning) new things.</div>
<div>
<br /></div>
<div>
Of the Tech Book Face Off posts, the top three most visited ones were <a href="https://sam-koblenski.blogspot.com/2019/02/tech-book-face-off-python-for-data.html"><i>Python For Data Analysis </i>Vs.<i> Python Data Science Handbook</i></a>, <a href="https://sam-koblenski.blogspot.com/2019/10/tech-book-face-off-how-to-design.html"><i>How To Design Programs </i>Vs. <i>Structure and Interpretation of Computer Programs</i></a>, and <a href="https://sam-koblenski.blogspot.com/2019/05/tech-book-face-off-seven-concurrency.html"><i>Seven Concurrency Models in Seven Weeks</i> Vs. <i>CUDA by Example</i></a>. These were all good, but the one that was most fun for me was easily <a href="https://sam-koblenski.blogspot.com/2019/11/tech-book-face-off-game-engine-black.html"><i>Game Engine Black Book </i>[<i>Wolfenstien 3D </i>Vs. <i>Doom</i>]</a>. Reading these books was a blast, and writing down my thoughts about them was just as fun and satisfying. I'm surprised it didn't gain more traction, but I'm pretty much done trying to figure out which posts are going to take off and which ones will fly under the radar of the Internet.</div>
<div>
<br /></div>
<div>
For the coming year, I'm planning on keeping up the same cadence, which means 18 more posts. That's really 18 posts, too. I checked. I didn't get to that exciting blog series I alluded to last year because the schedule was filled up with reviews, but I'm intending to start in on it right away this year. The book review posts will be much reduced as well. I've only got about a dozen programming books on my list that I'd still like to read, and I'll probably hold off on them while I work on this other project.</div>
<div>
<br /></div>
<h4>
Technical Books</h4>
<div>
I really dug into the technical books this past year, even more than the previous year, and I just met my goal of 22 books (plus 2 pop physics books) while working through most of my tech book backlog. As I had hoped, most of these books were quite good. A fair number of them even made it onto my read-again list. There were only a few duds and one stinker. Here's a run-down of them, roughly ranked in order of preference and linked to the longer Tech Book Face Off reviews.</div>
<div>
<br /></div>
<div>
<b>The Good</b></div>
<div>
<ul>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/11/tech-book-face-off-game-engine-black.html">Game Engine Black Book: Wolfenstein 3D</a></i> - Between the high level of nostalgia and the fascinating topic, there was no way this book wasn't going to top this list. It's an incredibly well-done guide to how Wolfenstein 3D was made.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/11/tech-book-face-off-game-engine-black.html">Game Engine Black Book: Doom</a></i> - Arguably, this book is even better than the Wolf3D one, but you should really read that one first so this one comes second. I highly recommend them both for anyone curious about how these legendary games were done on such feeble hardware.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/03/tech-book-face-off-rails-antipatterns.html">Rails AntiPatterns</a></i> - I loved the tactic this book took of showing bad Rails code, explaining why it's bad, and then showing how to fix it. Some books do this sporadically without telling you in advance, but this worked so much better, knowing that each example was intentionally bad from the start so there was no confusion about what was the right way to do things.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/06/tech-book-face-off-data-smart-vs-python.html">Data Smart</a></i> - This was the most fun I've had reading a book on Excel spreadsheets. Not that I read many books on spreadsheets, but if I did this would definitely be the best. It's about implementing data science algorithms in Excel with humor, and it's so much better than it sounds.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/09/tech-book-face-off-dont-make-me-think.html">Don't Make Me Think Revisited</a></i> - I enjoyed the first book, and the revised edition is just as good. Learn all about how to design user interfaces that make sense, and have a great time doing it.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/10/tech-book-face-off-how-to-design.html">Structure and Interpretation of Computer Programs</a></i> - A classic that still holds up today for teaching the fundamentals of programming and much more, this book has a steep learning curve, but the rewards match the effort it takes to get through it.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/04/physics-book-face-off-hidden-reality-vs.html">The Hidden Reality</a></i> - This was a super fun and mind-expanding read, taking us through the numerous types of multiverse concepts that have been thought up by cosmologists. Brian Greene continues his excellent, approachable writing style with this enjoyable book.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/04/physics-book-face-off-hidden-reality-vs.html">Parallel Worlds</a></i> - Michio Kaku does his own tour of the different types of multiverses we can conceive of, with a few more fantastical stories thrown in for good measure. This is another great book to read to get the high-level overview of this topic.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/09/tech-book-face-off-facts-and-fallacies.html">Facts and Fallacies of Software Engineering</a></i> - You don't have to agree with everything in a book for it to be excellent, and that's the case here with Robert L. Glass' thought-provoking arguments about the software engineering industry. Still relevant after 17 years.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/07/tech-book-face-off-programming.html">Professional CUDA C Programming</a></i> - If you're interested in GPU programming and want to play around with your nVidia graphics card, this book has all the information you need to get started in a nicely written, diagrammed, and organized guide.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/06/tech-book-face-off-data-smart-vs-python.html">Python Machine Learning</a></i> - It's a solid introductory text on the fundamental machine learning algorithms, both in how they work mathematically, how they're implemented in Python, and how to use them in scikit-learn and TensorFlow.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/04/tech-book-face-off-effective-python-vs.html">Data Science From Scratch</a></i> - Fundamentals are so important to learning a topic well, and this book does a great job of teaching the fundamentals of data science by implementing the algorithms from scratch in Python.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/05/tech-book-face-off-seven-concurrency.html">Seven Concurrency Models in Seven Weeks</a></i> - I've loved every 7-in-7 Weeks book that I've read, and this one is no exception. It's an entertaining read through seven different ways to do concurrent programming with today's technology.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/09/tech-book-face-off-dont-make-me-think.html">The Non-Designer's Design Book</a></i> - Learn how to design boldly in text and graphics with a few simple rules and clear, straightforward guidelines. Anyone and everyone who works around websites should give this quick read a look.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/07/tech-book-face-off-getting-clojure-vs.html">Getting Clojure</a></i> - If you're looking for a fun read and a tour of the Clojure programming language, this is the book to pick up.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/04/tech-book-face-off-effective-python-vs.html">Effective Python</a></i> - Every programming language has its beyond-the-beginner-level book on how to write programs in that language well, and this is the one to read for Python.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/08/tech-book-face-off-programming-elixir.html">Programming Elixir ≥ 1.6</a></i> - An excellent book for learning the ins and outs of this highly concurrent, fault-tolerant language, and it's well worth a read if you're operating in that domain.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/08/tech-book-face-off-programming-elixir.html">Metaprogramming Elixir</a></i> - This book fills in the few gaps in <i>Programming Elixir ≥ 1.6</i>, and it's a great companion to that book.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/09/tech-book-face-off-facts-and-fallacies.html">Programming Pearls 2</a></i> - This is a fairly decent algorithms book that's worth a read as a casual second or third book on programming algorithms.</li>
</ul>
<div>
<b>The Not-So-Good</b></div>
<ul>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/05/tech-book-face-off-seven-concurrency.html">CUDA by Example</a></i> - This book is clearly written, but not especially well-written. Still, it's a good introduction to CUDA programming that covers the basics. Just don't feel compelled to read it all the way through, as the later chapters are fairly useless.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/07/tech-book-face-off-getting-clojure-vs.html">Learn Functional Programming With Elixir</a></i> - Neither thorough on Elixir nor especially focused on teaching the unique aspects of functional programming, this book left a lot to be desired. It's fine, but not great in any respect.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/07/tech-book-face-off-programming.html">Programming Massively Parallel Processors</a></i> - For a subject that is inherently interesting to me, this book managed to be tedious, verbose, and opaque in its explanations, and way, way, way too long. The necessary information is in there, but it's not worth the effort when there are better options available.</li>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/03/tech-book-face-off-rails-antipatterns.html">The Rails 5 Way</a></i> - This book was so much longer than it had to be. It was light on discussion and super heavy on documentation, as if it was simply a transcription of the online documentation into a book.</li>
</ul>
<div>
<b>The Suck</b></div>
<ul>
<li><i><a href="https://sam-koblenski.blogspot.com/2019/10/tech-book-face-off-how-to-design.html">How to Design Programs</a></i> - Nearly 750 pages of the most tedious, drawn-out, agonizing explanations about learning to program, I disagree with the title given to this book. Nowhere in it does the reader learn how to design programs, maybe design of functions at best.</li>
</ul>
<div>
This showing is much better than last year, with a half-dozen more good books, and less bad books. It was actually quite hard to rank the good section beyond the first few because they were all well-written and engaging. I don't know if I was better at selecting good books to read, or if I just got lucky, but I was pleased with the high level of quality in this year's book list.</div>
</div>
<div>
<br /></div>
<h4>
Novels</h4>
<div>
As planned, I read even more technical books this year than last, but I still managed to read some great novels in the past year. Partly, this is because of listening to audiobooks in the car, and I have a 30 minute commute to fill. The number of books I got through was still less, but some of them were much longer than what I read the previous year. Again, they're roughly ranked.</div>
<div>
<ul>
<li><i><a href="https://www.amazon.com/Way-Kings-Stormlight-Archive-Book-ebook/dp/B003P2WO5E">The Way of Kings</a>, <a href="https://www.amazon.com/gp/product/B00DA6YEKS">Words of Radiance</a>, <a href="https://www.amazon.com/gp/product/B01NAWAH85">Oathbringer</a></i> - Wow. Just WOW. I could not believe how wonderful these books are. The characters are all flawed and broken and so human, the story drags you along as you watch in amazement at how everything develops, and the world that Brandon Sanderson built in this Stormlight Archive series is simply incredible. It has weight. It has life. It has history. I can't wait for the next book to come out.</li>
<li><i><a href="https://www.amazon.com/Fifth-Season-Broken-Earth-Book-ebook/dp/B00H25FCSQ">The Fifth Season</a>, <a href="https://www.amazon.com/gp/product/B01922I1GG">The Obelisk Gate</a>, <a href="https://www.amazon.com/gp/product/B01N7EQOFA">The Stone Sky</a> </i>- This trilogy is just as awesome as the Stormlight Archive series, and I was constantly amazed at how clearly written the world of the Stillness is. The fantastical powers that the orogenes and Guardians have could be confusing, but N. K. Jemisin writes so simply and beautifully that everything was crystal clear in my mind as I read it. The story is at the same time one of the most engrossing tales I've ever read and a powerful allegory about the struggles of racism in society. It's enlightening and revealing without being accusatory. Beautifully done.</li>
<li><i><a href="https://www.amazon.com/gp/product/B000FC1ICM">The Golden Compass</a>, <a href="https://www.amazon.com/Subtle-Knife-His-Dark-Materials-ebook/dp/B000FC1KJS">The Subtle Knife</a>, <a href="https://www.amazon.com/gp/product/B000FC1GJW">The Amber Spyglass</a></i> - Throughout this trilogy I was continually surprised by the incredible imagination of Philip Pullman. Each book brings entirely unique new elements into an already rich and diverse world, or rather many-worlds. Even with all of these new elements being introduced, the world always felt cohesive and real. Everything made sense within the context of the story. This is a hard thing to do right, and Pullman did it masterfully.</li>
<li><i><a href="https://www.amazon.com/Eye-World-Book-Wheel-Other-ebook/dp/B002U3CCYM">The Eye of the World</a></i> - I'm just getting started in this long <i>Wheel of Time</i> series, and I'm already hooked. The first book is essentially one long, thrilling chase through a world of mystery and magic. The reality of the world is revealed slowly, and the suspense of wondering when you'll find out that next tidbit of knowledge about the world was gripping. By the end I have more questions than answers, and I'm ready to learn more about the Wheel of Time.</li>
<li><i><a href="https://www.amazon.com/Snow-Crash-Novel-Neal-Stephenson-ebook/dp/B000FBJCJE">Snow Crash</a></i> - This book was ridiculously fun, plain and simple. The setup makes no sense at all and the story doesn't care one whit about anything, but it doesn't matter. You're a pizza delivery boy in the future who also happens to be the world's best samurai swordsman and an elite haxxor. You get mixed up in some crazy shit and lots of weird stuff happens. It's a wild ride, and you're just going to have to read the book to see how it all works out.</li>
<li><i><a href="https://www.amazon.com/Jurassic-Park-Novel-Michael-Crichton-ebook/dp/B007UH4D3G">Jurassic Park</a></i> - It's a book about man recreating dinosaurs in the modern age. What could go wrong? And what's not to like? This book is basically a classic at this point, and great fun to read.</li>
<li><i><a href="https://www.amazon.com/Ringworld-Larry-Niven-ebook/dp/B01513ZIL6">Ringworld</a></i> - While the premise of this book was interesting—humans and aliens go visit an enormous world built in a ring around the aliens' host star—I just couldn't get into this book by Larry Niven. The story was only tangentially about the ringworld, and the main focus was actually about whether people could be bred for luck. It seemed like Niven wanted to write about building a ringworld, but couldn't figure out how to write a compelling story around that so he had to also write about this other thing to justify it. Also, his writing was too disjointed for my tastes. Scenes changed so abruptly and dialog and narration was so terse that I had trouble understanding what was going on most of the time. That was a huge disappointment, especially considering the other books on this list.</li>
<li><i><a href="https://www.amazon.com/dp/B000W9399S">The Color of Magic</a></i> - This book was another disappointment, but for a different reason than <i>Ringworld</i>. The writing was fine, even funny sometimes, but the discworld made no sense at all. Completely random and nonsensical things would happen to the main characters at every turn, and I never could figure out what the plot was about. It didn't take long to lose interest in the characters altogether, since whatever happened to them wouldn't make any sense whatsoever and they were probably going to end up fine anyway. There were a lot of similarities in style to <i>The Hitchhiker's Guide to the Galaxy</i>, and while similar elements somehow worked there, they fell totally flat in <i>The Color of Magic</i>. That leaves me with no reason to read the other 40 books in the series, lucky me.</li>
</ul>
<div>
One of my goals here was to branch out and read new authors, and I mostly held to that goal. There's no Stephen King, Neil Gaiman, or Dragonlance books on the list, but I did read another book by Neal Stephenson after <i>Seveneves</i> and was not disappointed. The first eleven books were extremely hard to rank. They're all basically equivalent levels of awesome in my mind, and you absolutely should go read them if you haven't already. The stories are incredible, and the worlds these authors imagined and built are even more incredible. </div>
<div>
<br /></div>
<div>
I also reread <i>Jurassic Park</i>, from my youth, by way of introducing it to my daughter, who absolutely loves dinosaurs. It was well received. For this year, I already know I'll be rereading <i><a href="https://www.amazon.com/gp/product/B000FC1J76">The Lost World</a></i> with her. I'll also be continuing <i>The Wheel of Time</i> saga, and starting another new author for me, Ursula K. Le Guin, with <i><a href="https://www.amazon.com/Wizard-Earthsea-Cycle-Book-ebook/dp/B008T9L6AM">A Wizard of Earthsea</a></i>. Other than that, I'm thinking of finishing up the <i>Foundation</i> series by Isaac Asimov and reading a few more new authors. All I know is there are a lot more worlds to explore.</div>
</div>
<div>
<br /></div>
<h4>
Video Games & Movies</h4>
<div>
I actually have not watched many movies this year. It didn't seem like there were too many worth watching. I did see <i>Avengers: Endgame</i>, of course, and it was probably one of the best movies I've ever seen, considering the vast context and buildup of the rest of the MCU. I also finally saw <i>Captain Marvel</i> (I wait until I can get movies from the library), and that was pretty good, too. </div>
<div>
<br /></div>
<div>
One of the more unique movies that I did end up watching was <i>Annihilation</i>. I happened to get it when I was by myself in the house one night, and I popped it in the PS4, turned off the lights, and turned the sound way up. That was an intense, wonderfully creepy experience. Do you know how great the sound effects are in that movie? And that freaking bear, holy crap. I'm surprised I got any sleep that night. </div>
<div>
<br /></div>
<div>
I also rewatched <i>The Lord of the Rings</i> with my wife. Those movies hold up really well. If the movie drought continues, I've got a growing list of other (not-so) old movies to watch again, like <i>Jurassic Park</i>.</div>
<div>
<br /></div>
<div>
As for video games, I'm still enjoying the <a href="https://sam-koblenski.blogspot.com/2014/12/all-in-good-fun-learning-with-lego.html">LEGO videogames</a> with the kids. We finished up <i>Jurassic Park</i> (are you seeing a theme here?), <i>The Hobbit</i> (much better than the movies), and most of <i>Marvel Super Heroes</i>. We're still working on <i>The LEGO Movie</i>, <i>Batman 3: Beyond Gotham</i>, and <i>LEGO City: Undercover</i> (this game is ginormous). Then we've started <i>Marvel Super Heroes 2</i>, <i>Marvel's Avengers</i>, and we just got another batch of LEGO games for Christmas. I don't know why, but I don't ever seem to get tired of these games. It's gotta be the character development. (I'm kidding; it's the graphics.)</div>
<div>
<br /></div>
<div>
For myself, I ended up playing <i>Lightning Returns: Final Fantasy XIII</i> and <i>God of War III</i>. These are two very different games, and yes, they're both older than dirt. I said last year my backlog was deep. I know <i>Final Fantasy XIII</i> got a lot of criticism, but I still enjoyed the whole trilogy. I especially liked <i>Lightning Returns</i> for the new battle system. Gone are the menus and inventory lists, and instead you link four different actions to each of three different outfits to set up your available commands for a battle. You have a few other special actions that you can do, and then you're choosing actions in real time during battles. It's a much more dynamic, exciting battle mechanic than selecting actions from menus, and like most Final Fantasy mechanics, surprisingly deep. It really made the game for me. Lord knows the plot didn't.</div>
<div>
<br /></div>
<div>
<i>God of War III</i> was another installment of hack-n-slash, vengeance-upon-the-gods action game that was pure entertainment. I was reminded of how perfectly responsive the controls are in these games, and I think the most fun to be had was beating the crap out of Hercules.</div>
<div>
<br /></div>
<h4>
The Year Ahead</h4>
<div>
I have more than enough games to play this year as I play through my backlog. I actually got more new games than I finished, so that backlog is just getting deeper. If only there was more time. I won't be reading as many technical books, so there's that, but I'll probably fill a lot of that time with more novels and the next blog project. Still, maybe I can squeeze in a couple more games, and I haven't picked up the guitar again, yet.</div>
Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-79760239661437820262019-12-16T20:45:00.000-06:002019-12-21T16:19:24.283-06:00Sharpen Your Programming ToolsProgramming is like any other craft, whether that be engineering or woodworking or auto repair. Every craft has its tools that must be learned and maintained in order to do beautiful work and make wonderful things. In the craft of programming, one of the main tools we work with is programming languages. The more languages we know and the better we know them, the more versatile and valuable we can be as programmers and the better our solutions become. Fortunately, it's extremely easy to find websites to help you practice with programming problems and build up your skills with the languages you already know or bring you up to speed on languages you're trying to learn. They're so easy to find that it may be a bit overwhelming to pick one and settle in to actually work on some problems. Here are some of my favorite (free) sites—and why I like them—to help simplify that decision.<br />
<br />
<a name='more'></a><h3>
Monoglot Programming Puzzle Sites</h3>
<div>
These sites focus primarily or exclusively on one programming language. They tend to be great for getting started in that language or practicing the basics of a language that you're not that strong in, yet. However, they can also go deep into their chosen language because of the focus on a single language.</div>
<div>
<br /></div>
<h4>
<a href="https://rubymonk.com/">RubyMonk</a></h4>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://rubymonk.com/"><img alt="RubyMonk screenshot" border="0" data-original-height="320" data-original-width="953" height="133" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhk1gjQ0WJY8ZSKRSZfl7l0WWhzV7Vuyc6GqL8rWhpT-pvacky0AbnA4NPQlgSuEq5igpGtXRyjIJXVNpGZ9H0xi-cc7V_eD8N0VzrDXxVGhpOs97G4vpa7cWqKBEM0a7eYsqv3-9MJSo8/s400/rubymonk.png" title="" width="400" /></a></div>
<br />
RubyMonk is a nicely designed site that introduces Ruby and leads you on a journey through the language's various features from objects to metaprogramming. It's part tutorial with explanations of all of the language concepts that it covers, and part practice with additional exercises to solve after you've read the material. Throughout the tutorial are lines of code that you can run to see what they do, and other places where you're asked to enter your own code to accomplish certain tasks to participate more in the lesson. These input areas also act as scratch pads where you can experiment with the Ruby language to your heart's content. It's a great, interactive way to learn the language, and one of the better implementations of this concept that I've seen. I particularly enjoyed the Zen master and apprentice theme throughout. It was quite calming and relaxing, making for a pleasant learning and practicing experience.</div>
<div>
<br /></div>
<h4>
<a href="http://rubykoans.com/">Ruby Koans</a></h4>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://rubykoans.com/"><img alt="Ruby Koans screenshot" border="0" data-original-height="456" data-original-width="1008" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSJrzaG15Ehex-CdnDoBFrrzvGOk0y8MweDd5CxeY6horfgt_xN65MDDVAtC9SW1fwUhuGLU3Og8jq2rU5qjH_db4Q7cU6Kwq1N2w9Bkbc_bcg4M3lJNAUulS5LFrRk4QEKKlK4LbwGSk/s400/rubykoans.png" title="" width="400" /></a></div>
<br />
Continuing with the Zen theme, this Ruby practice site takes you away from the internet by having you download a set of files that are set up as Ruby test files. After making sure that Ruby is installed, you simply run each file in Ruby and see what the output looks like. It will tell you how the first test in the file fails, and then you have to go and fix the code to make it pass. You continue in this way along the path to enlightenment. It's pretty slick, and the progression of the problems was well thought out and nicely done. Whereas RubyMonk has the advantage of having everything work in the browser, Ruby Koans has the advantage of making you work in a more realistic programming environment, and its focus on the testing culture of Ruby is an added benefit.</div>
<div>
<br /></div>
<h4>
<a href="http://www.pythonchallenge.com/">The Python Challenge</a></h4>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.pythonchallenge.com/"><img alt="Python Challenge screenshot" border="0" data-original-height="402" data-original-width="638" height="251" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTcHXnfx6XiPdP0ZG867hmcIEiDiWbp1SohLlCs8oibF5wadPfe_Nwi-yQja_agFaXbcruSa8J43jiMcRoL77QGKWJB0zkR0WV7zKUM3cYgUMCZCI1US9RmlSo-ZJuv7fTOgmzJkolDmg/s400/pythonchallenge.png" title="" width="400" /></a></div>
<br />
This is probably the most clever of the sites in this list. The Python Challenge starts off with a couple fairly simple puzzles, but then quickly ramps up the difficulty level. For each puzzle you're given an image with a clue, and you have to figure out how to edit the URL to advance to the next level. This set of puzzles is extremely entertaining. I felt like Indiana Jones, or maybe Benjamin Gates from <i>National Treasure,</i> while trying to solve these puzzles, putting the clues through different Python functions and coming up with different ways to manipulate them in order to find the right next step. The site even helps you out with extra hints when you're on the right track but haven't fully solved the puzzle, yet. It's very well done, and I highly recommend giving it a try. Don't go looking for how to solve the problems, either. It's much more satisfying to do it yourself. Like Ruby Koans, you're own programming environment is required.</div>
<div>
<br /></div>
<h4>
<a href="http://www.4clojure.com/">4Clojure</a></h4>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.4clojure.com/"><img alt="4Clojure screenshot" border="0" data-original-height="579" data-original-width="1146" height="201" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtl8HW405c6AKcgitc5u_MQkfdHn2XRxRFVGokBYZCri_bcX4dBsgs4SfKePGaSmB9loFc793quAbpa25HtCHBi7bpkXJ8a5eERdHIhFsrhHRE_Q_qRnc-pKxeokmWT9VY3a9FL2J-KpM/s400/4clojure.png" title="" width="400" /></a></div>
<br />
<span id="goog_2142494307"></span><a href="https://www.blogger.com/"></a><span id="goog_2142494308"></span>This site is a straight up problem solving site for the Clojure programming language. Each of the 156 problems has a set of tests that should pass if you fill in the right code in the editor and run the tests. The output when your code fails is pretty sparse, so you'll either need a local Clojure environment or an online REPL (the link on the site is dead) for debugging and experimenting. The problems range from simple fill-in-the-blank problems for learning syntax to fairly difficult mathematics and algorithmic problems. Once you've solved any given problem, you can look at how other people have solved that same problem, but only for people that you've followed on the site. It's easy to choose a dozen or so people that have solved all of the problems from the site's Top Users list, and scanning through others solutions is a great way to learn new tricks in any programming language. We'll see this feature pop up in a number of the other sites here. I especially liked 4Clojure's simple, clean interface and the nice set of puzzles they have to solve with a good range of difficulty for learning the language. They do have some bumps in the road, with some problems seemingly out of logical order if you proceed through them numerically. Some earlier problems require syntax and functions that you learn about in later problems. The elementary, easy, medium, and hard problems are all mixed together, too. But, for the most part it works well, and solving the medium and hard problems was quite satisfying.</div>
<div>
<br /></div>
<h4>
<a href="https://sites.google.com/site/prologsite/prolog-problems">99 Prolog Problems</a></h4>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://sites.google.com/site/prologsite/prolog-problems"><img alt="99 Prolog Problems screenshot" border="0" data-original-height="560" data-original-width="1150" height="193" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaT_7aGTJNd3p0b7mjiqtOooYpWp-MAQ28ExhXUF35bpAKHP5CJKYW1SRn9r3qiqXzrAuH8Xchkk1KC1Cj0mWkPEDRPVr0qnwIh0m8YIxnVq7-Ea0HyO9yRbh-dXJyskgVNU2GvZgBOmY/s400/99prologproblems.png" title="" width="400" /></a></div>
<br />
If you want to try your hand at Prolog, this is a great site for doing just that. I would imagine these problems could be solved in a similar manner in Erlang and possibly Elixir as well, but beyond that, we're probably getting too far from a logic language for the problems to be the kind of challenge that they were intended. Come to think of it, miniKanren—a logic language built on top of Clojure—may also work here. Anyway, this is a great set of problems for buffing up your logic programming skills with a nice progression in material and difficulty, and there are solutions for if you get stuck or just want to see if there was another way to solve the problem. It's another bring your own programming environment site with just the bare-bones problems and solutions provided. The meat of this site is in the problems themselves.</div>
<div>
<br /></div>
<h3>
Polyglot Programming Puzzle Sites</h3>
<div>
These sites allow the user to solve problems in numerous different languages, sometimes any language they want to use is fair game. The nature of these sites tends to focus on the problem solving or puzzle nature of programming as a result, since general problems can be solved in any language.</div>
<div>
<br /></div>
<h4>
<a href="http://rubyquiz.com/">Ruby Quiz</a></h4>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://rubyquiz.com/"><img alt="Ruby Quiz screenshot" border="0" data-original-height="490" data-original-width="806" height="242" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBR_ObflPRxPOi_QSONJVorNExtC57_PFxWpL9DmrnLYPZC6lBKbsan9te3CBpFgYOYqo70U9KZkhT3-cbg1qRN0XtwwYJiJpHG7v5sSXN70Py-FVAbYcp4Asn4nMYxlEx9PT9418gI1c/s400/rubyquiz.png" title="" width="400" /></a></div>
<br />
Okay, what gives? I say this is going to be the polyglot section and the first site I list is another Ruby site. Well, Ruby is in the title, but it's not at all restrictive. You need to use your own programming environment, so you can solve the problems in whichever language you want. The solutions on the site are given in Ruby, but the result of running the program for each problem is known, so you'll know when you've got the program working in any other language. What's nice about these problems is that they're more like miniature real-world programming problems instead of the standard textbook exercises of most sites. This site was run with a new problem once a week for three years before switching management over to a different person and a mailing list, so there are 156 problems here to solve. That's still enough to stay busy for quite a while, so have fun exploring the standard libraries of your favorite programming language to efficiently solve these problems.</div>
<div>
<br /></div>
<h4>
<a href="https://www.codewars.com/">Codewars</a></h4>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.codewars.com/"><img alt="Codewars screenshot" border="0" data-original-height="768" data-original-width="1366" height="223" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7mxxlOFZoLm-zJKyCTlEAI_0MK7WO0GXoMcrjvJe3iLNC-TefzPShs8QOImrQ7rc-crfarDFpLs00NTzbQOAu5mIgr_JcEbE3IWdGIwInFRR2LWchzSTTV4g-vYn5FVpFqeuAuEerFV8/s400/codewars.png" title="" width="400" /></a></div>
<br />
Codewars is one of the more fully-featured sites on this list. It has huge sets of programming puzzles, called kata, for about 20 different languages, and the number of kata is constantly growing because users can submit their own kata for others to solve. The kata are ranked by difficulty level, and you get different amounts of experience for solving different levels of kata. There are no specified tracks to the kata; it's somewhat arbitrary what order you'll solve them as you're free to choose the next kata at every step. As you gain experience, or honor, you advance up the ranks from 8kyu to 1kyu. It's a nice gamification that keeps you going to solve more kata at higher difficulty levels, if the challenge and satisfaction of solving programming puzzles wasn't enough already. In addition to the huge selection of puzzles, you can look at other users' solutions, and discuss them through comments on the site. It adds a nice social aspect to the puzzle solving as long as you can keep the conversations civil, as we all should do in programming debates.</div>
<div>
<br /></div>
<h4>
<a href="https://exercism.io/">Exercism.io</a></h4>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://exercism.io/"><img alt="exercism.io screenshot" border="0" data-original-height="481" data-original-width="1119" height="171" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGkjHJQJFOp1ty3FQD7XOHKSFpBJT8blHeaxaaA0tWKuOzFNjuvM9A0wc5trdN5MZrboO8xZ7vOxJLcKNEbzJkDfJRYG97yyKFccBozkhhvHVK9SCHGuiifKuNJYBxZbwEHAPG4Qiz52A/s400/exercism.png" title="" width="400" /></a></div>
<br />
Exercism.io is similar to Codewars in that it supports a ton of different languages, and there's a social aspect to looking at other users' solutions and commenting on their code. Here it's encouraged to critique other people's code in a constructive and respectful way in order to try to help your fellow programmers improve their skills. Through your analysis and others critiques, you should also improve. It works pretty well, too. Compared to Codewars, there's less puzzles for each language, although more are being added all the time, and Exercism.io takes a serial approach to solving the problems. It's also done offline rather than on the website, so you'll download a little script to get started that pulls down the first problem for whichever language you want to practice. You solve the problem in your own environment, check it against a set of tests, and submit it with the script from a terminal. Then you can go on the site and review other people's solutions. It's a different way of doing what Codewars does from your own computer, and it works surprisingly well.</div>
<div>
<br /></div>
<h4>
<a href="https://projecteuler.net/">Project Euler</a></h4>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://projecteuler.net/"><img alt="Project Euler.net screenshot" border="0" data-original-height="444" data-original-width="1016" height="173" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEij62jZOz0f3CpJ0fh3QiiaZqfgcidmtwmql4OHTRbHVuy9fiQMfHEdSQIb39jaZnV4XBRpAEVzfbk6Q3Ttdwo4gUDLGg0X9kQPb1ocgaJugPjIFB-GVQlUzW035OWLRyJdqvkvFq9t96Q/s400/projecteuler.png" title="" width="400" /></a></div>
<br />
<span id="goog_2142494329"></span><a href="https://www.blogger.com/"></a><span id="goog_2142494330"></span>This puzzle site is the essense of a programming puzzle site. The problems are simply math problems that you could solve by hand, if you wanted to, but trust me, you don't want to. Take the first problem as an example: Find the sum of all the multiples of 3 or 5 below 1000. I could calculate this answer by hand, but honestly, I'm going to write code to do it. You can pick any language you want of course, and when you've solved it, just submit the answer to the site. Your time from seeing the problem to submitting the solution is logged, and you can go to the forum for each problem to discuss it and see how others have solved it. Many problems aren't even solvable on a computer if you just try to brute-force a solution. They require additional thought to make the code efficient enough to solve the problem before the Earth gets swallowed up by the Sun. On the other hand, there are some problems for which closed form solutions can be found, so keep your eye out for those! This is a challenging and engaging site, and it's ever so satisfying to solve a particularly difficult problem in an elegant way. They're always adding more problems to the list, too. It's currently up to 683 problems, so get crackin'!</div>
<div>
<br /></div>
<h4>
<a href="https://programmingpraxis.com/">Programming Praxis</a></h4>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://programmingpraxis.com/"><img alt="Programming Praxis screenshot" border="0" data-original-height="514" data-original-width="1028" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjryVR8szwxqDY0N3ju1fHdLK7FU-Xb0RhOMYtXZZNUhf1faD7pgciDDf03g_Sq0gQiTKSBqwTv6whTAli9ex0SVmwNNGPKGG1pzUlK93EOUrHyITbtH20TsHvIeb_MRamKTbZiRAlm7cs/s400/programmingpraxis.png" title="" width="400" /></a></div>
<br />
This site is a running list of programming exercises with new ones added on Tuesdays and Fridays every week. The exercises are intended to be solved with Scheme, and that's the language the solutions are given in. But of course you can solve the exercises in any language you like. This site has been going strong since 2009, so there's plenty of exercises to choose from. The chronological list only goes through 2013, with 491 exercises there, but they keep going after that, up to the present day. You can either go back through the history and try to solve them all, or use the site to consistently practice the most recent exercises starting today. Either way, there's a ton of stuff here to keep you busy and help you improve your programming skills in any language you want.</div>
<div>
<br /></div>
<div>
<br />
Well, there you have it, ten of my favorite sites for sharpening your programming tools. They run the gamut from introductory tutorials for specific programming languages to huge lists of fun, challenging puzzles for any language under the sun. There are tons more sites like these out there, probably many more great ones that will help you become a better programmer. The important thing is to find a couple sites that have engaging enough puzzles to hold your attention and help you level up your skills, and keep at it. Putting in that effort will noticeably improve your programming abilities in addition to being rewarding in its own right. I know I'll be working (playing?) through a few of these sites for a long time to come.</div>
Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-32327281020782792632019-11-25T21:26:00.000-06:002019-11-25T21:26:10.612-06:00Tech Books I Will Read AgainI have read a crap-ton of technical books, mostly on software, but some either more general or hardware related so I felt the need to generalize the genre to "technical" books. If you've been following my blog for the last year, you'll agree that especially recently, my reading rate has been arguably excessive. I'm reaching a point where I'd like to slow down and focus on some other things in my free time, but I'm also reflecting on all of the great and not-so-great tech books I've read. One of the defining factors in whether I think a tech book is excellent versus merely good is if I have the urge to read it again. (For anyone wondering, there is no distinction needed for the bad tech books.) This feeling might happen right after I finish it, or even while I'm reading it the first time. It also might take a while to percolate and rise back to the surface as a book I want to go back to. The bottom line is, a mark of a great tech book is that it's worth revisiting, so what follows is a list of tech books I've read that I thought were so great that I'm going to read them again.<br />
<br />
<a name='more'></a><h4>
<a href="https://sam-koblenski.blogspot.com/2013/02/tech-book-face-off-design-patterns-vs.html">Object Oriented Design Heuristics </a></h4>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEge2HAnPRs-K0_D-kdjSlZgHpZ2F8FTPMYQwdJaNz2_wZz2NdkT9J-uppvXUy0zXtxgaZNtISwtHf8yiirhy5m7YgdsbqC0HBhYEND1pY2kLBLNcspZJ1VhP0R7yFbAGwJ-JdgNEy2YWv8/s1600/object_oriented_design_heuristics_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="388" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEge2HAnPRs-K0_D-kdjSlZgHpZ2F8FTPMYQwdJaNz2_wZz2NdkT9J-uppvXUy0zXtxgaZNtISwtHf8yiirhy5m7YgdsbqC0HBhYEND1pY2kLBLNcspZJ1VhP0R7yFbAGwJ-JdgNEy2YWv8/s200/object_oriented_design_heuristics_cover.jpg" width="155" /></a><br />
<br />
This is a completely underrated book about software design, not in the sense that it gets poor reviews, because it doesn't, but it's not a very well-known book for how great it is. Other software design books, specifically <i>Design Patterns</i>, the notorious GoF book, steals all of the oxygen in the room, but I strongly prefer <i>Object-Oriented Design Heuristics</i>. While GoF is heavily prescriptive and mechanical in how it lays out the design patterns to use when writing software, OODH digs into the guidelines and rules-of-thumb that lie underneath those patterns. Instead of attempting to memorize a couple dozen patterns and their various applications, the reader learns why certain ways of organizing code work and naturally make the code easier to understand, debug, and change. It teaches fundamental concepts instead of lists of tools, and in the end knowing the fundamentals is much more valuable. In addition to being able to derive the tools you need without needing to remember them all, you can apply the fundamentals to new situations and invent new tools when the ones you have don't fully meet the requirements. The fundamentals are always worth revisiting.<br />
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2013/03/tech-book-face-off-pragmatic-programmer.html">The Pragmatic Programmer: From Journeyman to Master</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQ77chwgEu8sipobJCeIjPMYapiQvhwSsrGyOdDoukfvztZ2GYZTjBjgzf4VFXPA0mmiaphL9a087BpDjVTLNvuSlRPORxDhScwL0uU5XsdCPCPCIaGtEBhxTnOS5NAa-7c6bKRQAZJmo/s1600/pragmatic_programmer_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="382" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQ77chwgEu8sipobJCeIjPMYapiQvhwSsrGyOdDoukfvztZ2GYZTjBjgzf4VFXPA0mmiaphL9a087BpDjVTLNvuSlRPORxDhScwL0uU5XsdCPCPCIaGtEBhxTnOS5NAa-7c6bKRQAZJmo/s200/pragmatic_programmer_cover.jpg" width="152" /></a></div>
<div>
<br /></div>
<div>
<i>The Pragmatic Programmer</i> was a shoe-in for this list. It is the definition of short and sweet for a programming book, and every piece of advice contained within it is pure gold. So many concepts that I use every day are contained in these pages. DRY. Tracer bullets. The Broken Window Theory. It's all in here, and now there's the 20th anniversary edition. It's the perfect excuse to go back and read it again for the first time. </div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2014/07/tech-book-face-off-clean-code-vs-agile.html">Clean Code: A Handbook of Agile Software Craftsmanship</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8V61xTmbZ0KnYUXR4nZa2fHPu8YTtxpFGVsM-TnBQDy1jbFu8T51pQhRAWp_mfItj59woLfg3SYfjS3AJmDCTXWHsLsQ-PUJPT04Kcme7-RO6RdSD2irPOFtobk7dAR8GQQXQC1wYkSs/s1600/clean_code_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="387" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8V61xTmbZ0KnYUXR4nZa2fHPu8YTtxpFGVsM-TnBQDy1jbFu8T51pQhRAWp_mfItj59woLfg3SYfjS3AJmDCTXWHsLsQ-PUJPT04Kcme7-RO6RdSD2irPOFtobk7dAR8GQQXQC1wYkSs/s200/clean_code_cover.jpg" width="155" /></a></div>
<div>
<br /></div>
<div>
<i>Clean Code</i> has a fair amount of overlap with the previous two books, but there's plenty that's unique in here and Robert C. Martin is such a great writer that it's worth it to reread this book, too. This book was surprisingly engaging for how deep it went into the minutiae of writing code. Like <i>Object-Oriented Design Heuristics</i>, it lays out guidelines and heuristics for writing better code, but it focuses a bit more on the specifics of how to name variables, structure functions, and write comments. It may veer more into the list-of-tools arena, but Martin reasons about everything nicely and it just seems to work for me in a way that GoF doesn't. It's always good to periodically refresh the concepts behind writing clean code.</div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2014/07/tech-book-face-off-clean-code-vs-agile.html">Agile Principles, Patterns, and Practices in C#</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6E4-gT5EOjufsnPM1TqqehyphenhyphenYjRZXwWSAEX6LON6bWpgz4mW0sA7uNQ1Q-0xQRfr9oaPwqXW-pkokn4LvuVdmvQBf0xHvD31wPlqNBHkPAUc08vf4F64MtVO4Kt-2HaBGgpcYv_Fn839A/s1600/agile_principles_patterns_practices_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="374" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6E4-gT5EOjufsnPM1TqqehyphenhyphenYjRZXwWSAEX6LON6bWpgz4mW0sA7uNQ1Q-0xQRfr9oaPwqXW-pkokn4LvuVdmvQBf0xHvD31wPlqNBHkPAUc08vf4F64MtVO4Kt-2HaBGgpcYv_Fn839A/s200/agile_principles_patterns_practices_cover.jpg" width="149" /></a></div>
<div>
<br /></div>
<div>
This book covers much more ground than any of the previous three books, and it is subsequently much longer than any of them. It weaves together the topics listed on the cover quite well throughout the book, driving many of the chapters with simple, instructive examples. This book is where the SOLID design principles are laid out, along with a review of most of the standard patterns in GoF. Where this book does things better than the prescriptive GoF is in how the pattern discussions are example-driven and grounded in the design principles covered earlier in the book. The flow is so much better, and the reader comes away with a great understanding of why the patterns work, when to use them, and how to apply them appropriately.</div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2019/09/tech-book-face-off-facts-and-fallacies.html">Facts and Fallacies of Software Engineering</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjG3qe3J2v4tgsmQ1JbvrsT9qSyD-YBnByyEOIzFw3HdJnizQe2z_jmrDwZUzUCnyE5nYELYxFQLQIYhl3MKbVKoIMFuq1H5TesKm8NZfYRcGZFVEUWUQ-E3nebnlDuqF9X4SqySRPXhgk/s1600/facts_and_fallacies_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="399" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjG3qe3J2v4tgsmQ1JbvrsT9qSyD-YBnByyEOIzFw3HdJnizQe2z_jmrDwZUzUCnyE5nYELYxFQLQIYhl3MKbVKoIMFuq1H5TesKm8NZfYRcGZFVEUWUQ-E3nebnlDuqF9X4SqySRPXhgk/s200/facts_and_fallacies_cover.jpg" width="159" /></a></div>
<div>
<br /></div>
<div>
This is such a quick, sharp, and relevant read, that it's definitely worth a reread every now and again. You won't agree with every one of Robert Glass' facts and fallacies, (when does anyone agree with everything in a software engineering book?) but they will make you think and rethink your assumptions, which is always valuable for growth as a developer. </div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2016/06/tech-book-face-off-c-programming.html">The Little Schemer</a> and <a href="https://sam-koblenski.blogspot.com/2018/12/tech-book-face-off-seasoned-schemer-vs.html">The Seasoned Schemer</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKrbe6MUlu54d0rjzFpj0wWM9OiMLNy2H4RizHhMUwQbqYxBzhvgu5mpcf9zvkdoGUEJHpQckAMu233OKZeEsQ6Skv5pQCRygYo1APtn7Rb8Qg4-YOCnBY2TmEl3FOdMk-s1HXXURVJ58/s1600/the_little_schemer_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="373" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKrbe6MUlu54d0rjzFpj0wWM9OiMLNy2H4RizHhMUwQbqYxBzhvgu5mpcf9zvkdoGUEJHpQckAMu233OKZeEsQ6Skv5pQCRygYo1APtn7Rb8Qg4-YOCnBY2TmEl3FOdMk-s1HXXURVJ58/s200/the_little_schemer_cover.jpg" width="149" /></a><img border="0" data-original-height="499" data-original-width="384" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7fcNGEw6iDXR4djZXkSwVE8e8ODN4jyI_S-mQ0odXYLWuWskOcR0M-xFyivoEHnCR5yGcjebasSDGMRAE7LYmz0A2L75wB4a4MRbK5_pTVbmLhjjdM1UT3klNiq09lq7b7TJvqvYxBgQ/s200/seasoned_schemer_cover.jpg" style="text-align: center;" width="153" /></div>
<div>
<br /></div>
<div>
These gems were such a joy to read the first time, there's no way they were not going to make this list. The quirky humor and self-guided Q&A format work perfectly. I can't think of a better way to refresh my basic programming skills with Scheme, and rewrite a Scheme interpreter, than to reread these books. Scheme is essentially a fundamental programming language, so in addition to these books being great to go back to, Scheme is a great language to go back to and polish up any rusty programming skills.</div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2019/10/tech-book-face-off-how-to-design.html">Structure and Interpretation of Computer Programs</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8FDgg9qc8X023N2BS54Uhk1MB75jwTspJs7oaME2mTgS8eBPB1Opsp4FYYzXmCZuDHqUhW_ig73Ry4gJhDsBWZyGOycoyGrdf2VAI1jLq3j2ej8LHxODaP20P8jOVBY8yQR9sbMy9lVQ/s1600/sicp_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="333" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8FDgg9qc8X023N2BS54Uhk1MB75jwTspJs7oaME2mTgS8eBPB1Opsp4FYYzXmCZuDHqUhW_ig73Ry4gJhDsBWZyGOycoyGrdf2VAI1jLq3j2ej8LHxODaP20P8jOVBY8yQR9sbMy9lVQ/s200/sicp_cover.jpg" width="133" /></a></div>
<div>
<br /></div>
<div>
While SICP is the polar opposite of <i>The Little Schemer</i> and <i>The Seasoned Schemer</i>, it's just as good for learning and reviewing Scheme and programming fundamentals. Most introductory programming books are not worth revisiting because they tend to focus on language syntax and mechanics, but that's not true here. Scheme doesn't have much syntax to begin with, so SICP goes deep into fundamental programming concepts and, well, how to structure programs. It goes so deep that it's probably only worth revisiting the first three of its five chapters (how often do you need to review how to build a register machine simulator?), but that still presents a challenging and valuable exercise.</div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2013/11/tech-book-face-off-ruby-programming.html">Eloquent Ruby</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheXrSp7H0jdag9aqGFcr4bIx8IEHBPvB3OAUmC1zT8-RPElyenOxEIdIM-0cnvQZGyZ_mMoBHxzTAibaFpC9A_whsYpWVoBBNNA4kLuj27YkXNp1NWVWuvigxEeUE_Y9_QHWt3L_7ePr4/s1600/eloquent_ruby_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="389" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheXrSp7H0jdag9aqGFcr4bIx8IEHBPvB3OAUmC1zT8-RPElyenOxEIdIM-0cnvQZGyZ_mMoBHxzTAibaFpC9A_whsYpWVoBBNNA4kLuj27YkXNp1NWVWuvigxEeUE_Y9_QHWt3L_7ePr4/s200/eloquent_ruby_cover.jpg" width="155" /></a></div>
<div>
<br /></div>
<div>
If I had to pick a favorite language, it would be Ruby, and this book exemplifies why. Everything needed to write beautiful Ruby programs is contained in here, and it has none of the boring cruft of an introductory programming language book. Russ Olsen is also an excellent technical writer, so the book is an enjoyable and easy read. This is how all intermediate-to-expert level programming books should be, and it makes it a pleasure to refresh those Ruby programming skills.</div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2018/09/tech-book-face-off-confident-ruby-vs.html">Confident Ruby: 32 Patterns for Joyful Coding</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjab9Yqa5hBujFXK9wup9yLxSZZ700meN1OsPwEV3RBM8byaKATe7b4LDr_eE6iCqAu2-LgRa2FqXNedT0SquhwgVdVfALrIqTQOdt3Si-lER_39eFfBRzwxnXvSQmXa_kPNlazZPSGcLg/s1600/confident_ruby_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="500" data-original-width="386" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjab9Yqa5hBujFXK9wup9yLxSZZ700meN1OsPwEV3RBM8byaKATe7b4LDr_eE6iCqAu2-LgRa2FqXNedT0SquhwgVdVfALrIqTQOdt3Si-lER_39eFfBRzwxnXvSQmXa_kPNlazZPSGcLg/s200/confident_ruby_cover.jpg" width="154" /></a></div>
<div>
<br /></div>
<div>
This book is like an up-to-date version of <i>Object-Oriented Design Heuristics</i> for Ruby. It's a quick, enjoyable read, with Avdi Grimm writing in an approachable, conversational style. The content is superb, detailing the best ways to write Ruby methods that have a definitive purpose without stumbling over edge cases that often weaken code. It's a great book for reviewing how to write clean, concise, purposeful code, and it would probably work for any programmer, not just the Rubyists out there.</div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2019/03/tech-book-face-off-rails-antipatterns.html">Rails Antipatterns: Best Practice Ruby on Rails Refactoring</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7yjwf_TPWaaZNj9PgUa1m7-ku7VPSHphwQ_Xis2sTL-Dd79Z3hk2StpTwl87mb-fKcglh-gh3guiGsFoWvoTJx_xm15iW53BNK8gJNpz7heOGtiEyLALx4hlrBkmBSAzOtl-PA3nTEgw/s1600/rails_antipatterns_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="383" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7yjwf_TPWaaZNj9PgUa1m7-ku7VPSHphwQ_Xis2sTL-Dd79Z3hk2StpTwl87mb-fKcglh-gh3guiGsFoWvoTJx_xm15iW53BNK8gJNpz7heOGtiEyLALx4hlrBkmBSAzOtl-PA3nTEgw/s200/rails_antipatterns_cover.jpg" width="153" /></a></div>
<div>
<br /></div>
<div>
The books on this list are here because they offer timeless advice, and this book is no different. It may seem like it's specifically about Ruby on Rails 3 programming, but the methods of refactoring the example antipatterns covered in the book—and why those antipatterns are bad in the first place—extend well beyond Rails. Bad code sucks for the same reasons no matter which programming language it's written in, and clean code shines in any language for the same reasons. It just so happens that Ruby and Rails are such pleasant vehicles for learning how to write clean code, and this book in particular uses a great style of showing how <i>not</i> to write code and how to fix that ugly code that was written before learning all of this great advice. </div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2013/04/tech-book-face-off-html-css-design-and.html">HTML and CSS: Design and Build Websites</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwGSocyU2oweEs5MPhL77mR7iG53-iuqxFaAu8SIEc22QcfqAFsOrRbW_P4Kfj61XiA6zceTgnZ6AqqbtsJjTfpd5eClzG__2dvfEF_ieKYjnnPon86zVIoPKSc7YuRoBfH2DlrjJ4Vwg/s1600/html_and_css_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="398" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwGSocyU2oweEs5MPhL77mR7iG53-iuqxFaAu8SIEc22QcfqAFsOrRbW_P4Kfj61XiA6zceTgnZ6AqqbtsJjTfpd5eClzG__2dvfEF_ieKYjnnPon86zVIoPKSc7YuRoBfH2DlrjJ4Vwg/s200/html_and_css_cover.jpg" width="159" /></a></div>
<div>
<br /></div>
<div>
This is hands-down the most beautiful technical book I've ever read, and after looking through it, it becomes obvious that this is the best way to present the HTML and CSS syntax. These are the languages of visual design for the web, after all, so it makes sense to show how all of the tags and attributes work with full color pictures and diagrams on every page. It's especially helpful when showing the differences between borders, padding, and margins in CSS, but really most HTML tags and CSS attributes translate well to this kind of presentation. It's also quick to page through every once in a while, and such an enjoyable experience that it's worth doing multiple times.</div>
<br />
<h4>
<a href="https://sam-koblenski.blogspot.com/2019/09/tech-book-face-off-dont-make-me-think.html">Don't Make Me Think (Revisited)</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoArTjgiCCVcCuwl5_XL6FvSFpxV8ULKrDm92JGJBIsGQmNDCtdwGAsNDUxVILQwj1ddUUk000BPCuR8w03b5VbE8YwSEluJ2cYtJ9UJOdm4EBy2A3E8zKbtGe-JrZJ9oWV8rNAlSOxrk/s1600/dont_make_me_think_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="389" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoArTjgiCCVcCuwl5_XL6FvSFpxV8ULKrDm92JGJBIsGQmNDCtdwGAsNDUxVILQwj1ddUUk000BPCuR8w03b5VbE8YwSEluJ2cYtJ9UJOdm4EBy2A3E8zKbtGe-JrZJ9oWV8rNAlSOxrk/s200/dont_make_me_think_cover.jpg" width="155" /></a></div>
<div>
<br /></div>
<div>
This book was an easy addition to this list, partly because I already have read it twice. It was just as good the second time around when the updated version came out. <i>Don't Make Me Think (Revisited) </i>is packed with examples of both the right way and the wrong way to design websites, but mostly the right way. Steve Krug has a quick wit and the full color page layouts are great to look through for ideas. All of this makes the book a fast, easy read so there's no excuse to not refresh your knowledge on website design best practices.</div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2015/05/tech-book-face-off-envisioning.html">Envisioning Information, Visual Explanations, and The Visual Display of Quantitative Information</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjg17thltr_OiWrrvR4WxpHCnUyn9WcnZkHWce5eMGnwB82NT3zPTJaLTuQXtome9MbRJEQwaBKehzxO904wuzMLgPF_ey-06k30hzl36YZ8yFexDLiu56r-gwgMrHseAXSnDh2A8NJEAo/s1600/envisioning_information_cover.jpeg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="325" data-original-width="260" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjg17thltr_OiWrrvR4WxpHCnUyn9WcnZkHWce5eMGnwB82NT3zPTJaLTuQXtome9MbRJEQwaBKehzxO904wuzMLgPF_ey-06k30hzl36YZ8yFexDLiu56r-gwgMrHseAXSnDh2A8NJEAo/s200/envisioning_information_cover.jpeg" width="160" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7UgYL4d_NcgRSELdeSs88fkrlIDSd9aINtQgMo5PlRb-qbey2OYCTYBI1K0aM8DIZ4W2HP-qN5ZUwWW_MpvTx1BSDmdcgqMm-czLiD4p18q1-ycf0Kb3emEdOksXPJuNppC6sCHvPYJE/s1600/visual_explanations_cover.jpeg" imageanchor="1" style="clear: left; display: inline !important; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="330" data-original-width="260" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7UgYL4d_NcgRSELdeSs88fkrlIDSd9aINtQgMo5PlRb-qbey2OYCTYBI1K0aM8DIZ4W2HP-qN5ZUwWW_MpvTx1BSDmdcgqMm-czLiD4p18q1-ycf0Kb3emEdOksXPJuNppC6sCHvPYJE/s200/visual_explanations_cover.jpeg" width="157" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhotAuCCNTmWdhCPSm4b-cEyXfRDKTCUVHSXswEsrVrJVdKS2tmVhVka8j7fsEx1VH6V9hev5tYL4d5AQ8LwklUW1HLUdZWI2s1B7M-45PdVTD43n9TEcxYRTks78WMNhVJbtN8djbSUjI/s1600/visual_display_cover.jpeg" imageanchor="1" style="clear: left; display: inline !important; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="321" data-original-width="260" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhotAuCCNTmWdhCPSm4b-cEyXfRDKTCUVHSXswEsrVrJVdKS2tmVhVka8j7fsEx1VH6V9hev5tYL4d5AQ8LwklUW1HLUdZWI2s1B7M-45PdVTD43n9TEcxYRTks78WMNhVJbtN8djbSUjI/s200/visual_display_cover.jpeg" width="161" /></a></div>
<div>
<br /></div>
<div>
This is another set of books that's easy to kick back with and peruse at your leisure, this time with a focus on how to display information in a way that makes insights jump off the page. These books are full of excellent advice and ideas on how to present information in charts, diagrams, and pictures so that the relevant information is clear and obvious instead of confusing and obfuscated. How data is represented is as important as the quality of the data itself, and reviewing these books will help drive your imagination to show that data in the best light for consumption and the spread of ideas.<br />
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2019/06/tech-book-face-off-data-smart-vs-python.html">Data Smart: Using Data Science to Transform Information Into Insight</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGvyaC8RzP2HRCaiffpHJpEjKdXRJU5RDcyheQhLqrhyWK6Y92eiv6uc8kuqcXC0-K98N9vIHxjTYv1Mk-RlqPqAoEdIAMwvehTsOSTKQtKgKi1h650mtE_RJCeO2NJ4gCDegQod8i0Sk/s1600/data_smart_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="399" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGvyaC8RzP2HRCaiffpHJpEjKdXRJU5RDcyheQhLqrhyWK6Y92eiv6uc8kuqcXC0-K98N9vIHxjTYv1Mk-RlqPqAoEdIAMwvehTsOSTKQtKgKi1h650mtE_RJCeO2NJ4gCDegQod8i0Sk/s200/data_smart_cover.jpg" width="159" /></a></div>
<div>
<br /></div>
<div>
I enjoyed this book way more than I was expecting to. John Foreman has a great sense of humor that really comes through in his writing, and he's able to take a normally dry topic and make it, dare I say, entertaining. He runs through a bunch of data science algorithms using real data and everyone's favorite spreadsheet program, Excel. Really. He develops all of these algorithms in Excel, and yes I meant entertaining. It's not a typo. A spreadsheet is actually a natural fit for learning about these algorithms because you can see every step in the process all laid out before you in black and white. It was so good, I'm planning to read the book again the next time I need a refresher course on K-Means Clustering or regression models. </div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2019/04/tech-book-face-off-effective-python-vs.html">Data Science From Scratch: First Principles With Python</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgETAUIxUf9Qw5pIieXF_0dJQwK1jfa67Q4YSuGkCe5FoSiNSRvjJ0KzQ6-o0HcBDjx26CKc9bv1ju5q3K1Ycie_SrBpOBE79oWtWqlqPoE52pmJKZpFx3ey1z2BlW27PzBHRhETqGsT3c/s1600/data_science_from_scratch_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="381" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgETAUIxUf9Qw5pIieXF_0dJQwK1jfa67Q4YSuGkCe5FoSiNSRvjJ0KzQ6-o0HcBDjx26CKc9bv1ju5q3K1Ycie_SrBpOBE79oWtWqlqPoE52pmJKZpFx3ey1z2BlW27PzBHRhETqGsT3c/s200/data_science_from_scratch_cover.jpg" width="152" /></a></div>
<div>
<br /></div>
<div>
This is another great book about data science that teaches the reader how to implement a number of data science algorithms and supported with an excellently dry wit. Instead of using Excel as <i>Data Smart</i> did, this time we're doing everything in Python. More than anything, rereading this book would be for the sake of deliberate practice. It's great for practicing programming, problem solving, and algorithmic knowledge because the book is building up data science from first principles. The more comfortable you are with the fundamentals, the easier everything else gets.</div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2019/06/tech-book-face-off-data-smart-vs-python.html">Python Machine Learning</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAzFI2IPIdP6clEw3ZYHqbSfL8AUs05xCtWsTvDWzBwrRH4uQJvKI9gCA8OIYN8agMrRn_DyYtZZK2vluC1aGWVG0qlkXQkzvQ1itYSoVSC32UWhp5XdTri4hCgayWq9Z8pHKfKFO8nM8/s1600/python_machine_learning_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="500" data-original-width="406" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAzFI2IPIdP6clEw3ZYHqbSfL8AUs05xCtWsTvDWzBwrRH4uQJvKI9gCA8OIYN8agMrRn_DyYtZZK2vluC1aGWVG0qlkXQkzvQ1itYSoVSC32UWhp5XdTri4hCgayWq9Z8pHKfKFO8nM8/s200/python_machine_learning_cover.jpg" width="161" /></a></div>
<div>
<br /></div>
<div>
Okay, it may start to be feeling like this list is getting a little data science and machine learning heavy, but there's a reason for that. These are great books for developing analytical thinking and problem solving skills in the context of programming. Most programming books are either introductory books that aren't worth reading again once you know the language, or they're programming craftsmanship books—like the first part of this list—that are definitely worth reading, but don't flex the analytical parts of your brain much. These machine learning books do work your analytical brain more, and that goes for <i>Python Machine Learning</i>, too.</div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2019/07/tech-book-face-off-programming.html">Professional CUDA C Programming</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKm0PFe3mWEKr0Ct5oMtaXmJVaqLdmjH76Pw2Rhoq5OgLBZWmMs4dSh8gv8VqJxKXcdmH_co6pI8zrDlQ9UU7MAKhR3MQ_fCPdgu9Svfp5PFnuOk5QsBaHiHM_gYTt2Tl0zrqTzzbgU9M/s1600/professional_cuda_c_programming_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="399" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKm0PFe3mWEKr0Ct5oMtaXmJVaqLdmjH76Pw2Rhoq5OgLBZWmMs4dSh8gv8VqJxKXcdmH_co6pI8zrDlQ9UU7MAKhR3MQ_fCPdgu9Svfp5PFnuOk5QsBaHiHM_gYTt2Tl0zrqTzzbgU9M/s200/professional_cuda_c_programming_cover.jpg" width="159" /></a></div>
<div>
<br /></div>
<div>
I found multiprocessor parallel programming with CUDA to be fascinating, and this was the best of the three books I read on the subject. It was well organized and understandable with nice, clear writing on a complex topic. It will be excellent practice to work through this book again.</div>
<div>
<br /></div>
<h4>
<a href="https://sam-koblenski.blogspot.com/2016/01/tech-book-face-off-godel-escher-bach-vs.html">The Annotated Turing</a></h4>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJIFnCB00eKvLLNWyjYgWG1uoKQrI5HLqNYfREIpp_GSxruRR4_bgGgS30a3l1r6EENXUNedu4esHZWeEX5oo1fo7qOQS8J3f2Ou5lXkv-jSsMvMDlyez9dKaXcaeZDDpLc2p9d5Jazfk/s1600/the_annotated_turing_cover.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="499" data-original-width="333" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJIFnCB00eKvLLNWyjYgWG1uoKQrI5HLqNYfREIpp_GSxruRR4_bgGgS30a3l1r6EENXUNedu4esHZWeEX5oo1fo7qOQS8J3f2Ou5lXkv-jSsMvMDlyez9dKaXcaeZDDpLc2p9d5Jazfk/s200/the_annotated_turing_cover.jpg" width="133" /></a></div>
<div>
<br /></div>
<div>
If we're going to talk about the fundamentals, we can't get more fundamental than Alan Turing's paper on computability. Charles Petzold did an amazing job parsing out Turing's paper and making it accessible to more people. This is still a challenging read, but incredibly rewarding and thought provoking. Of all the books on this list, this book will be the one that I get the most out of with a second reading. The topics covered here are so deep and subtle that it would be foolish to think that one reading would be enough to absorb everything sufficiently. I'm looking forward to another round with this book in the near future.</div>
<div>
<br /></div>
<h4>
Notable Omissions</h4>
<div>
People may notice some popular programming books missing from this list. First, I'll say that this is my list, and I'm not making any apologies for it. Wanting to read a book more than once is some of the highest praise I'll give for a book. Reading a book a second time is a rare occurrence. There are so many potential other good books out there yet to be read! Having said that, here are a few books I purposely didn't include.<br />
<i><br /></i>
<i>Code Complete</i>, I felt was too long and dry for my tastes. I much preferred <i>Pragmatic Programmer, Clean Code, and Agile Principles, Patterns, and Practices in C#</i>, and between the three of those books the same topics were covered in a more enjoyable way and even in less pages!<br />
<br />
<i>Refactoring</i> was just a slog to get through. I thought it was worse than GoF in its itemized drudgery. Much of the same material is present in other books on the list, but those books teach the reasons and motivations behind good refactorings without listing them out ad nauseum.<br />
<br />
<i>Introduction to Algorithms</i> maybe should have been on the list because I did read this book twice, but the second time was such an exercise in tedium with not much reward that I wouldn't recommend it to anyone other than aspiring CS professors. There have got to be more accessible algorithms books out there to brush up on this subject.<br />
<br />
<i>Seven Languages in Seven Weeks</i> is a book that I would put on any best tech book list because it's an excellent book that's definitely worth a read, but probably only once. I don't have much desire to read it or any of the other <i>Seven in Seven Weeks</i> books again, even though I thoroughly enjoyed them all on the first read.<br />
<br />
Well, there you have it. A complete list of tech books that were so good that I'm planning to read them again. They all have a common thread running through them in that they teach the fundamentals of whatever topic they're covering, and they do it really well. The fundamentals are extremely important in any field, and it's worthwhile to constantly revisit them and refresh your skills. Doing that with books that are engaging and enjoyable to read makes the whole process that much easier, and that is another defining characteristic of the books on this list. I imagine I'll add a few more books over time, but probably not too many. It's a rare thing to find a tech book worth reading multiple times.</div>
Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com2tag:blogger.com,1999:blog-6227791252039855587.post-19089841469141543382019-11-04T20:27:00.000-06:002019-11-04T20:27:54.542-06:00Tech Book Face Off: Game Engine Black Book [Wolfenstein 3D Vs. Doom]After all of the heavier reading I've been doing lately—machine learning, CUDA programming, fundamental Lisp programming, etc.—I wanted to kick back and read something a bit more relaxing and entertaining. Luckily, at just the right time a friend lent me a couple of books that promised to fit the bill perfectly: the <i>Game Engine Black Books</i> for <i><a href="https://www.amazon.com/Game-Engine-Black-Book-Wolfenstein/dp/1070515841">Wolfenstein 3D</a></i> and <i><a href="https://www.amazon.com/Game-Engine-Black-Book-DOOM/dp/1099819776">Doom</a>, </i>both by Fabien Sanglard. I grew up with these games, with them being my first and second PC FPS games. I played countless hours of these and other id Software games and other games that used id Software engines like Rise of the Triad, Heretic, and Hexen. I couldn't wait to dig into these books and see what was underneath the games that pleasantly wasted away the night hours of my youth.<br />
<br />
<table><tbody>
<tr><td><a href="https://www.amazon.com/Game-Engine-Black-Book-Wolfenstein/dp/1070515841"><img alt="Game Engine Black Book: Wolfenstein 3D front cover" height="275" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimD2LIrog9RtZ7HM1tw47L-lJQVbQ7soG6eL2NnWp7KhCv6ORHXorSV0ZxAXN-86uWWJLSgonT6LUCbNSBZwvdEgMZaajSBpMRekn3mIQJ8njQuz5m-GiXEwW-XOL37_gwH9FYdS5qA3w/s1600/game_engine_black_book_wolf_3d_cover.jpg" /></a></td><td style="padding: 30px;">VS.</td><td><a href="https://www.amazon.com/Game-Engine-Black-Book-DOOM/dp/1099819776"><img alt="Game Engine Black Book: Doom front cover" height="275" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYjNd-yYUy7kJWmhVhv7CS26gLLWwNjDKlIOcioKakrfZwpG2X5lWaXB0NTGWPuofL1H_zD434i1gQJoEWYmJNOZyZaWjZ40jPku015Alfy0fIKb42uGpdq4DCcvtLpp5w3lByovefXgA/s1600/game_engine_black_book_doom_cover.jpg.jpg" /></a></td></tr>
</tbody></table>
<a name='more'></a><br />
<h4>
Game Engine Black Book: Wolfenstein 3D</h4>
I quickly realized what a beautiful book this was going to be with nearly full-page, glossy color pictures of the game and die photos of the Intel 80386-DX processor that ran the game when it first came out on May 5th, 1992. It was incredibly fun to just page through the book and look at the different pictures, and peruse the various explanations surrounding the pictures about the game engine's design. It took a little while to settle down and actually start reading through the book.<br />
<br />
This Black Book starts out with forwards by John Carmack, Tom Hall, and John Romero, and then there's a short introduction that gives an overview of the game market, the PC market, and the state of video games in that era. This was the time of Super Nintendo and Sega Genesis, and the i386 processors had been out for a few years by the time Wolfenstein 3D was released. The i386DX-33 was over four times more powerful than the Super Nintendo processor, and even more powerful i486 processors were coming to the market from mid-1989 to 1992. The video game consoles were primarily sprite-based hardware designs, so they weren't set up for running a 3-D game like Wolfenstein well at all. The PC was definitely the hardware of choice for this game, power-wise, but as Sanglard goes on to explain, it had its own downsides for game development.<br />
<br />
The next chapter went into the state of the hardware that was targeted for Wolfenstein 3D in detail. The game was designed to run okay even on an i286 CPU, but even on the i386 processors, the compromises that were made in the design were astonishing. I had forgotten how bad things were back then with no hardware floating point units, only a handful of registers, and an utter mess of a memory hierarchy with real mode and segmented addressing. The state of video cards was not much better with no direct double-buffering in hardware, an essential feature for running smooth animation in a 3-D game. These VGA video cards were not set up to run games in the least, so double-buffering needed to be hacked and kludged together. Thankfully, that was possible to do. Sound was just reaching a tolerable level of performance at the time with AdLib and Sound Blaster cards, if you sprang for the add-in card and were able to get it working. Sound cards were rarely included in PC builds at the time, and I remember many a night fiddling with vaguely understandable IRQ and DMA settings before plug-n-play. Funny, now I actually know what those things mean, and what they're used for, but it's all handled automatically by the chipset and the OS.<br />
<br />
That chapter was definitely a fun trip down memory lane. The next chapter switched gears and was all about the decidedly small team that made the game, especially by today's standards, and they completed the game in four months! There were four full-time people that founded id Software a year before starting Wolfenstein 3D. John Carmack and John Romero were the programmers, Adrian Carmack was the artist, and Tom Hall was the creative director. Another four people contributed, but did not work on the game full-time from start to finish. These were Jay Wilbur in business, Kevin Cloud as a computer artist, Robert Prince as the composer, and Jason Blochowiak as another programmer. The development studio was none other than the first floor of John Carmack's two floor apartment. Wild. The rest of the chapter went through the development tools they used and built to make the code, level maps, artwork, and sound assets of the game.<br />
<br />
Chapter 4 gets into the real meat of the game architecture, and it's the longest chapter of the book. It was fascinating to learn about the ingenious performance hacks and algorithms they (mostly John Carmack) implemented to achieve playable frame rates with the hardware they had to work with. The game is restricted in numerous ways so that the code doesn't have to spend much time calculating things like clipping or player perspective. The floor and ceiling were solid colors, and all walls were orthogonal. The player couldn't look up or down, jump, or crouch, so walls were always perfectly vertical and the (obscured) horizon was always dead-center on the screen. The only polygons in the game were the walls, which were rectangles. Everything else was a sprite. The maximum view port was smaller than full-screen at 304x152 pixels, and it could be adjusted down from there if needed for a faster frame rate on slower hardware. These along with plenty of other restrictions made the game playable on the i286, although the best experience was still on an i386DX or better CPU.<br />
<br />
The other common optimization was to code critical loops and code sequences in x86 assembly. Quite a few code listings in the book are just lines of assembly code, mostly of moving values around in various registers. Dropping down to assembly was necessary because compilers at the time weren't the best at outputting even remotely optimized code, and a smart developer could arrange assembly code much better than any compiler could. This deficiency in the compiler shouldn't all be blamed on it, for x86 assembly was truly horrendous. I am at the same time in awe of John Carmack for doing what he did with the code and eternally thankful that I never had to do x86 assembly programming like that myself. Even today it's something to be avoided because the complexity is now nearly unmanageable and compilers have made huge strides besides, but oh my gawd, that assembly programming must have been the most awful, tedious exercise in memorizing and recalling useless instruction set trivia.<br />
<br />
This whole chapter was pure gold, covering everything from the renderer in detail to the enemy AI to the audio programming and sound effects. The writing wasn't always the greatest, but explanations were at least insightful, and I could always parse out what Sanglard meant, even with numerous typos and grammatical errors. Normally these kinds of mistakes would be obnoxious, but the book was so full of great pictures, diagrams, and information that was completely engrossing to me that I was willing to overlook that shortcoming.<br />
<br />
The last few short chapters were also quite interesting, and a nice way to wrap up the book. Chapter 5 described the sequel to Wolfenstein 3D, Spear of Destiny. Chapter 6 covered a number of ports to other systems, including Super Nintendo (heavily compromised), Jaguar, iPhone, and VR ports. Chapter 7 concluded with a little information about the successors to Wolfenstein 3D and where each of the designers and developers are now. The whole book was an excellent tour of a classic game that changed the video game industry forever, and if you're at all interested in the details of one of the games that started the FPS genre, you have to read this book.<br />
<br />
<h4>
Game Engine Black Book: Doom</h4>
<div>
This Black Book followed a very similar format to the Wolfenstein 3D book with the same result of an incredible experience digging through the game engine of one of the most influential PC games ever made. I went through waves of nostalgia as I paged through pictures of the Doom game world and explanations of how it calculated and rendered the intense game that consumed hours upon hours of my free time growing up. It was fascinating to see how all of it was implemented, and on such meager hardware, too! id Software achieved an incredible amount of performance and features with their idTech 1 engine that they created for Doom.<br />
<br />
The book starts off with forwards by John Carmack and Dave Taylor and a short introduction just to wet your appetite. Then it jumps into the hardware that Doom was designed for in chapter 2. Doom was able to run on an i386, but by 1993 when Doom was being developed, the i486 was becoming affordable and was over 2x faster than the equivalent frequency i386, so that was the new target system. Everything else was improving rapidly as well. Video cards took a big step up in performance with the VESA VL-Bus and tighter component integration on the cards. Sound cards became even more fragmented, unfortunately, but sound quality was improving by leaps and bounds. Networking was also becoming possible by this time with multiple ways to connect PCs for multiplayer co-op and deathmatch play. Finally, compilers had improved to the point where it wasn't necessary to do nearly everything in assembly. The Watcom compiler allowed id Software to code Doom almost entirely in C, freeing the developers to think at a higher level and implement more features more rapidly.<br />
<br />
Chapter 3 is entirely new because id Software used an entirely different development environment for Doom. Instead of developing everything on unstable DOS PCs, they took the chance on using expensive NeXT workstations. Even a fairly basic NeXTstation cost $4,995 in 1991, and a NeXTcube ran $12,395! It turned out to be worth it, though, because these workstations were rock-solid and allowed John Carmack and the other developers to make incredible progress instead of constantly fighting with crashing machines and a poor development environment. This chapter did a great job going through the architecture and benefits of the NeXT systems, and how id Software used them to full effect.<br />
<br />
The next chapter was about the team and tools, like chapter 3 in the Wolfenstein 3D book. Things had changed dramatically, with the team moving to Dallas, TX and growing to fourteen people by the end of development. The tools were evolving, too, with some of the character sprites and animation done using stop motion capture, and the map editor (DoomED) taking on new features to support the multitude of new capabilities in the game engine. Walls in the game no longer needed to be orthogonal, although they still had to be vertical for rendering to be fast enough. Floors and ceilings could change height to create steps, platforms, and other environmental features. Various kinds of traps and ambushes were also possible now. It's quite amazing how many features were added to idTech 1 when compared to the Wolf3D engine. These new features, especially the non-orthogonal walls and varying heights, required a more efficient data structure for the maps, and the BSP (Binary Space Partitioning) node tree was commandeered for the job.<br />
<br />
Nearly half of the book is taken up by chapter 5 on the idTech 1 game engine. This chapter is just a monster of awesome information about the engine. It goes through every detail from the game tic design and use of fixed-point arithmetic to sound propagation and enemy AI. The section on the 3D renderer was especially interesting and detailed. There are great explanations on how the environment was drawn with its additional complexity, texture mapping with perspective correction, sprite clipping and animation, and diminished lighting to give the game its intense horror movie feel. This renderer section in particular was so interesting that it made me want to go implement a bunch of the algorithms myself just to be able to recreate them and see them in action. It looks like it would be immensely satisfying.<br />
<br />
The last chapter described in fairly good detail how a number of ports of Doom were done on other game consoles. The console ports included the Atari Jaguar, Sega 32X, Super Nintendo, Sony PlayStation, 3DO, and Sega Saturn, and both the system architectures and Doom engine implementations are described. I was in high school during this era of the great console wars, although I never played Doom on any of them, just the excellent experience of the PC. I had forgotten how different these console architectures were, and some of the design decisions that went into those systems were truly remarkable...and strange. The sections on the Super Nintendo and PlayStation ports were my favorites. The Super Nintendo would have never been able to run Doom on its own, but with the extra power of the SuperFX chip that was used in Star Fox and a few other games, it was able to pull it off reasonably well. The PlayStation port was the most faithful of the ports, and even added some cool new features like colored ambient lighting. Of all of these systems, the PlayStation was exceptionally powerful for making great games. Even though other systems might have looked more powerful on paper, they all had disadvantages and unfortunate design decisions that handicapped them while the PlayStation was so easy to develop for that its potential could consistently be achieved in practice.<br />
<br />
I enjoyed this book as much, if not more than the Wolf3D Black Book. From the beautiful full-color pictures to the detailed explanations of technical feature implementations to the well-chosen code listings sprinkled throughout the book, this was an incredibly fun guided tour of a legendary game engine. I devoured this book, and through it all, I wanted to do two things: go back and play through Doom again and re-implement some of those rendering algorithms for myself. I can't think of higher praise for a book, so if you've ever played Doom and want to know how it works, you need to go treat yourself to this book and the Wolfenstein 3D Black Book, too.</div>
Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com0tag:blogger.com,1999:blog-6227791252039855587.post-7912955153822954212019-10-15T21:53:00.000-05:002019-10-15T21:53:41.737-05:00Tech Book Face Off: How to Design Programs Vs. Structure and Interpretation of Computer ProgramsAfter reading and reviewing dozens of books on programming, software development, and computer science, I've finally come to a couple of books that I should have read a long time ago. I'm not quite sure why I didn't read these two books earlier. Distractions abound, and I always had something else I wanted to read first. I still wanted to see what they had to offer, so here I am reading <i><a href="https://www.amazon.com/gp/product/0262534800">How to Design Programs</a></i> by Matthias Felleisen, Robert Bruce Findler, Matthew Flatt, and Shriram Krishnamurthi and <i><a href="https://www.amazon.com/Structure-Interpretation-Computer-Programs-Engineering/dp/0262510871">Structure and Interpretation of Computer Programs</a></i> by Harold Abelson, Gerald Jay Sussman, and Julie Sussman. As I understand it, these books are meant to introduce new students to programming so not reading them until now will probably make it difficult to accurately critique them from the perspective of the target audience. I'm still going to give it a try.<br />
<br />
<table><tbody>
<tr><td><a href="https://www.amazon.com/gp/product/0262534800"><img alt="How to Design Programs front cover" height="275" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRmeYz0MUL0VJPjINQLMrB_IMsK-id1dSv1rM_UrdagppQvXsNIAERZO4526kcjZ8Z1Mnnn4ROwviBexNquIagXPp1s_Xa3RPF-eRgTikjIB-wT4dU435vQUmj_opeiAJrTtUnFxUkGwI/s1600/how_to_design_programs_cover.jpg" /></a></td><td style="padding: 30px;">VS.</td><td><a href="https://www.amazon.com/Structure-Interpretation-Computer-Programs-Engineering/dp/0262510871"><img alt="Structure and Interpretation of Computer Programs front cover" height="275" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4eRnrWDpNah8ZmySNO13AjOd3e92CKGpDVFN40XrVdsdLYTWQdk0qCPeQ1LOj1V7789s2c8dO7lmWhHWAizunpjqOvSStuO5BFZUYpf9pq9laHrXIIhm8hyoBbFM0AO5VnDB33H3ThCI/s1600/sicp_cover.jpg" /></a></td></tr>
</tbody></table>
<a name='more'></a><br />
<h4>
How to Design Programs</h4>
While I have read many programming books as ebooks on my Kindle, this is the first book I've read as an <a href="https://htdp.org/2018-01-06/Book/">online book</a>. It's available in print version, but the online version looked nicely formatted and was heavily cross-linked, which was quite helpful. Also, since the book was right alongside my code editor, I could easily try out the problems and copy-paste code into the editor to run it.<br />
<br />
The programming language used in this book as the vehicle for learning was a variation on Scheme called BSL (Beginning Student Language) that had heavy restrictions on the language features available for use. For example, lists could not be constructed with <span style="font-family: "courier new" , "courier" , monospace;">(list <list of elements>)</span> or <span style="font-family: "courier new" , "courier" , monospace;">'(<list of elements>)</span>, instead needing to be built up from <span style="font-family: "courier new" , "courier" , monospace;">(cons <element> <list>)</span>. These and other features were added with BSL+, ISL (Intermediate Student Language), and ISL+ as the reader progressed through the book.<br />
<br />
I was not a big fan of this approach, since it was tedious to learn the wrong way first (or at least the way nobody actually writes code) and then learn the right way that also makes more sense. Starting with the reduced forms of the language didn't add anything to the explanations, and it mostly served as a point of frustration as the reader was forced through unnecessary tedium until the nicer language features were introduced. It was also not clear by the end that ISL+ was equivalent to the full Scheme programming language, so by the time the student reached the end of the book, they wouldn't even be sure that they had learned a real programming language.<br />
<br />
The book was quite ambitious, since learning how to design programs starting from square one actually involves learning three almost distinct things at once: the syntax of a programming language, how to write code, and how to program. The first task is about learning what a programming language is, what keywords and punctuation are available, and what it all does when put together in the correct structure. This task alone can be overwhelming for the new student. Then, the second task, learning to write code, involves taking a small, well-defined problem statement, thinking up a solution to it, and translating that solution into code that can execute in the newly learned programming language. Normally, once some part of the language has been learned, the first two tasks can be done together, and they support each other as the student practices them both.<br />
<br />
The third task, learning how to program, is a much deeper activity that takes a much longer time to become proficient. Learning how to program is not just about solving bigger problems that may not be self-contained or well-defined. It's about learning how to organize all of the required code to solve the problem in a way that makes that code easy to understand, is less likely to suffer from bugs, is testable (and tested), and hopefully is flexible and extensible. I'm not convinced that this book taught this much more advanced skill, or at least not thoroughly.<br />
<br />
The book starts out with a little prologue chapter entitled "How to Program." It gives a short introduction and a few examples of how to write some simple programs in BSL, and here the authors try to get the reader excited about what they'll be learning in the rest of the book. I had some misgivings about it. They have a strange way of trying to connect with the reader by disparaging various subjects that the reader is likely taking in school, or at least has fresh memories about, like physics:<br />
<blockquote class="tr_bq">
Physics?!? Well, perhaps you have already forgotten what you learned in that course. Or perhaps you have never taken a course on physics because you are way too young or gentle. No worries.</blockquote>
I don't understand this kind of writing. It's not cool and it's not helpful to pretend to joke along with students about how much physics or mathematics or any subject sucks. It drives me nuts. Why can't we encourage students to take more of an interest in these subjects by showing how the knowledge can be useful, interesting, and dare I say, fun?! Thankfully, these comments don't continue in the rest of the book, but it's still irritating that they subtly perpetuate this anti-learning bias. It ended up coloring my opinion of the book in a negative way.<br />
<br />
The rest of the book is split into six major sections with "intermezzos" between each section. I'm not quite sure why the intermezzos are there because they just seem to continue on with more topics that would fit in as additional chapters within the six sections, but that doesn't matter much. The first section introduces BSL features in more detail than the prologue, and it also lays out the recommended steps of a design process in chapter 3. These steps are touted as the steps for how to design a program, but they're really steps for how to design a <i>function</i>. The process is fine as far as it goes, but it doesn't really scale to large programs. This process is used and refined throughout the book.<br />
<br />
The first section only introduced language features that allow for fixed-size data, so the next section introduces lists and recursion. It's a whole five chapters on lists, including basically a whole chapter of problems for the reader to solve. I don't remember lists being quite so hard to understand that it would require five chapters to adequately get the point across, especially with lists being a fundamental data structure of Scheme. Sorry, BSL+. Part of the problem is that the authors seem to explain things in as many words as possible. The text ends up plodding along with slow, tedious explanations, some of which don't even make much sense.<br />
<br />
Another part of the problem of understanding in this book is the poor choice of variable names. When the authors are not using single-letter names (<i>shudder</i>), they're using names like <span style="font-family: "courier new" , "courier" , monospace;">alos</span>. What is <span style="font-family: "courier new" , "courier" , monospace;">alos</span>? Is it like, "Alos, poor Yorick! I new him, Horatio?" No, that's not right. That would be 'alas.' Instead, <span style="font-family: "courier new" , "courier" , monospace;">alos</span> means "a list of strings." But why not just use <span style="font-family: "courier new" , "courier" , monospace;">list-of-strings</span>, or even <span style="font-family: "courier new" , "courier" , monospace;">strings</span> if you're into the whole brevity thing. The point is, these super abbreviated and truncated variable names make things more confusing than they need to be, because you have to keep the translation to the full name in your head along with the meaning of the variable and the rest of the code. Using full words for variable names makes the code so much more readable and understandable. It's not like we're working under any space constraints with the text of the code, except for the constraints of our own working memory.<br />
<br />
The third section goes into detail on functions and how they enable abstractions that will make the programmer's life easier. Abstractions allow the programmer to solve low-level problems once and then think about harder problems from a higher level. It's a critical programming skill to learn. This section follows a similar format to the last one, with four chapters on functions done in excruciating detail, one of which is full of problems for the reader. We also advance to ISL for this section, and near the end we achieve the final level up to ISL+. Yipee! I remember hating when textbooks would introduce one way of doing things and then later contradict themselves and reveal the real way of doing it. This failing is worse with simplified languages, so I'm pretty tired of "student languages" by now.<br />
<br />
The next section covers intertwined data, which is a convoluted title for a section that doesn't have a strong theme. The chapters in this section range from introducing trees to building a simple interpreter to processing multiple lists in parallel. The fifth section focuses on recursion for five chapters, and here they make the distinction between structural recursion, which is based on scanning lists, and generative recursion, which is more general and doesn't use lists as the looping mechanism. The final section discusses accumulators that are used with recursion to enable calculating properties of data that requires keeping track of additional state during the recursion. It's basically passing extra state variables in the recursive call in order to calculate aggregate or cumulative values on the data. All of these sections continued to have chapters near the end that were filled with extra problems for the reader.<br />
<br />
This book was super long at nearly 750 pages—one of the longest programming books I've read in a while—and it did not seem like it covered enough ground to warrant that length. There were also 528 problems in total, so a huge amount of practice if the reader worked through all of the problems. Most of the problems were pretty decent, and they stayed relevant and reasonable for the material covered beforehand. But the book as a whole didn't hold up to its goal of teaching the beginner how to design programs. Learning how to program is a huge undertaking, and I don't believe it's possible to adequately cover that whole process in one book. On top of that, the level of discussion in much of the book was too complex for the beginner, and it would just as likely serve to confuse them as to teach them. Conversely, it doesn't seem to work well as a second programming book, either because it is so slow and tedious and long. By the end all we've learned is the basics of Scheme, lists, functions, and recursion. <i><a href="https://sam-koblenski.blogspot.com/2016/06/tech-book-face-off-c-programming.html">The Little Schemer</a></i> taught much more in a more entertaining way in less than 200 pages. I can't recommend slogging through this book instead.<br />
<br />
<h4>
Structure and Interpretation of Computer Programs (SICP)</h4>
<div>
This book was the textbook for the entry-level computer science course at MIT for a number of years. I, unfortunately, was unaware of it until after I had finished college (not at MIT) and had come across it mentioned in a blog post by Steve Yegge, I think. Ah, yes, <a href="https://sites.google.com/site/steveyegge2/ten-challenges">here it is</a>. Apparently, that's also where I got the idea to read <i>How to Design Programs</i>, but fortunately, <i>SICP</i> was a better recommendation. I also didn't have the same issues with the book's humor that Steve did. I didn't mind the silly names of the students in the exercises, (you always know that Louis Reasoner got it wrong; you just have to figure out how) and some of the other jokes were actually kind of funny:<br />
<blockquote class="tr_bq">
In testing primality of very large numbers chosen at random, the chance of stumbling upon a value that fools the Fermat test is less than the chance that cosmic radiation will cause the computer to make an error in carrying out a “correct” algorithm. Considering an algorithm to be inadequate for the first reason but not for the second illustrates the difference between mathematics and engineering.</blockquote>
I mean, come on. That's not half bad for a computer science joke. The majority of the book was not about fun and games, though. It's a serious book on introducing the computer science student to how programming languages—particularly Scheme—work.<br />
<br />
<i>SICP</i> is split into five fairly balanced chapters. Each chapter starts off easy with an introduction to the material covered in the chapter and more detailed explanations of the mechanics of Scheme or the interpreter or whatever is the topic for the chapter. As things develop, the difficulty ramps up until near the end you can feel your brain going numb and draining out of your ears. Then you get a breather at the beginning of the next chapter with another gentle introduction.<br />
<br />
The first chapter starts off with the requisite intro-to-the-language stuff that every book for a new programming language needs. After covering Scheme's operators and primitives, we move on to functions and immediately jump into recursion. By the end of the chapter we're learning about how to pass functions around as arguments and return values, and I wonder how an entry-level student could really grok all of this in a semester course. This is just chapter 1!<br />
<br />
Chapter 2 teaches us all about how to structure and process data in Scheme. Since the fundamental data structure in Scheme is the list, this means we're going to get very comfortable with list processing (which is how Lisp gets its name, see?). Between these first two chapters, we gain a thorough understanding of the foundations of Scheme and how to put together Scheme programs to do interesting things with data. Even after reading so many books on programming and practicing it in the field for a couple of decades, I was quite enjoying this "beginner" programming book.<br />
<br />
Helpful exercises are interspersed with the text throughout the book, generally at the end of each subsection, and they are quite well thought-out exercises. With 356 exercises in all, they provide a ton of practice to ensure that the reader is understanding the material. At first they seem to be somewhat random but standard fare, asking the reader to solve programming problems with rational and complex numbers and other such mundane mathematical problems. Then, near the end of chapter 2, we learn how to implement generic arithmetic operations that can automatically promote and demote arguments from one class of number to another. It's pretty slick, if somewhat impractical. I can't think of a system where this behavior would be necessary, but it's cool to get it working nonetheless.<br />
<br />
The next chapter kind of let the wind out of my sails a bit. The previous chapters had really exemplified the elegance of Scheme with beautiful functional programming, but now we had to learn about the mucky reality of objects and mutable state. This chapter introduces the <span style="font-family: "courier new" , "courier" , monospace;">set!</span> operations that allow variables to be changed in place instead of creating and returning new variables that are set with the <span style="font-family: "courier new" , "courier" , monospace;">define</span> primitive. The allowance for changing variable values enables the creation and maintenance of objects with state, and this complicates the analysis of program execution because now we have to deal with side effects. The authors did a nice job of explaining when objects are useful, because we don't want to use them for everything:<br />
<blockquote class="tr_bq">
The object model approximates the world by dividing it into separate pieces. The functional model does not modularize along object boundaries. The object model is useful when the unshared state of the “objects” is much larger than the state that they share. An example of a place where the object viewpoint fails is quantum mechanics, where thinking of things as individual particles leads to paradoxes and confusions. Unifying the object view with the functional view may have little to do with programming, but rather with fundamental epistemological issues.</blockquote>
The second half of the chapter continues on from objects with concurrency, which does not play nice with mutable state at all, and introduces streams in order to deal with that problem. Streams are a mechanism that enables lazy execution of functions on lists. Instead of performing all of the computations on a list at the time the processing function is called, the function will return another function that will do the computation on its corresponding list element at the time that element is needed to be read. It's wild and confusing at first, but working through the exercises helps clarify how it all works.<br />
<br />
Chapter 4 tackles the task that all Lisp books seem to reach eventually, and that is to write an interpreter. <i>How to Design Programs</i> did it. <i>The Little Schemer</i> did it. <i>SICP</i> does it to, but it doesn't simply stop with one interpreter. No, after the basic interpreter, we go on to write a lazy interpreter that does delayed evaluation. Then, we write another interpreter that does ambiguous evaluation, meaning the programmer can specify a problem and an input range for that problem, and the interpreter will perform a search to find a solution (or every solution) that satisfies the constraints of the problem. Think that's enough? Not now that we're on a role! The final interpreter extends Scheme to be a logic language similar to Prolog. You would think the previous ambiguous interpreter would be a good basis for this extension, but the text uses the lazy interpreter as the base instead. Extending the ambiguous interpreter is left as an exercise.<br />
<br />
Things are getting pretty mind-bending by now, so why don't we finish things off with something truly warped. The last chapter goes through implementing a register machine model in Scheme. What's a register machine? It's basically a model of a computer that uses fixed registers, a load-store memory model, and low-level operations to execute an assembly language. Then we need something to run on this register machine, so we modify the interpreter to run on top of this assembly language. Now let's step back and think about what we've done. We now have an interpreter that takes in Scheme code, spits out assembly code, and runs it on a model of a computer (the register machine); and this is all done <i>inside another Scheme interpreter running on a real computer</i>. Wat? Let's think again about what we've done:<br />
<blockquote class="tr_bq">
From this perspective, our evaluator is seen to be a universal machine. It mimics other machines when these are described as Lisp programs. This is striking. Try to imagine an analogous evaluator for electrical circuits. This would be a circuit that takes as input a signal encoding the plans for some other circuit, such as a filter. Given this input, the circuit evaluator would then behave like a filter with the same description. Such a universal electrical circuit is almost unimaginably complex. It is remarkable that the program evaluator is a rather simple program.</blockquote>
It's mind-blowing, really, but we're not done. The last part of the last chapter walks through building a compiler so that we can compile Scheme functions down to the assembly language of the register machine, and modify the register machine so that it can run the assembly code directly instead of it being fed from the interpreter. If that's not enough, the last two exercises are simply about writing a scheme interpreter in C and a compiler in C instead of what we just did in Scheme. Easy-peasy, right?<br />
<br />
While these last two chapters were fun and fascinating, they were quite a stretch for one book. The first three chapters plus the basic Scheme interpreter would have been enough for the learning experience. I'm not sure how much practical knowledge readers would get out of the rest of the interpreters, the register machine, and the compiler. The explanations became very mechanical and it felt like a major effort just to fit in the code listings and brief descriptions while still keeping the book around 600 pages. Beyond the issue of cramming a bunch of complex stuff in the last chapter and a half of the book, there are much better books out there on compilers and interpreters, like <a href="https://www.amazon.com/dp/B009TGD06W">the dragon book</a> or <i><a href="https://www.amazon.com/dp/B004S82O40/?coliid=I3S3CS3PXJ81RO&colid=2QM6EOB4CC49&psc=0&ref_=lv_ov_lig_dp_it">Writing Compilers and Interpreters</a></i>, that go into more detail and explain those details more thoroughly. Likewise for machine languages and computer architecture, if you really want to understand the underlying machine language and hardware, <i><a href="https://www.amazon.com/Computer-Architecture-Quantitative-Approach-Kaufmann-ebook/dp/B078MFDTX4">Computer Architecture: A Quantitative Approach</a></i> is excellent. Although, for a lighter introduction, <i><a href="https://www.amazon.com/Computer-Organization-Design-ARM-Architecture-ebook">Computer Organization and Design</a></i> might be a better place to start.<br />
<br />
That criticism notwithstanding, <i>SICP</i> is an excellent book on both how to write programs in Scheme and how to write a Scheme interpreter. It's a solid textbook for a second course in programming, but not a first course. I can't imagine most entry-level students would grok everything this book tries to present, so it would be good to have some other programming language under your belt before tackling this beast. Given its age, it's still surprisingly relevant as a programming textbook, and quite enlightening.<br />
<br />
<br />
<i>SICP</i> is far and away the better book in this face off. True, <i>How to Design Programs</i> is meant for a less technical audience, but I'm not convinced that it would be an appropriate book for non-programmers, either. Scheme is just not the right introductory language, and something like Python or Ruby would be a much better learning language. Taking that consideration out of the equation, <i>SICP</i> packs a ton more content into 150 less pages, and it goes in much more depth on both basic programming concepts like lists and functions, and advanced concepts like streams and interpreters. Did I mention it also uses way better variable names? The code is much easier to understand as a result, and it's only the complexity of the concepts later in the book that make that code difficult.<br />
<br />
Definitely pass on <i>How to Design Programs</i>, but if you're in the mood to level-up your fundamental programming knowledge, give <i>SICP </i>a look. If you're so inclined to read it online, check out <a href="http://sarabander.github.io/sicp/html/index.xhtml">this version at Sara Bander's GitHub site</a>. It's rendered in a beautiful Linux Libertine font that's an absolute joy to read on a screen, and the footnotes come up in text boxes when clicked. It's the best experience I've had reading an ebook.</div>
Sam Koblenskihttp://www.blogger.com/profile/16088649179174139591noreply@blogger.com2