Related to my last blog post regarding accessing setter methods of a prefab script, originally, I was accessing the script through a class field of type Transform, which was attached to a enemy bullet prefab. To get access to the script, I needed to call GetComponent from the returned reference by Instantiate. See the Gist below.
From what I've been told by internet anonymouses, GetComponent is an expensive call to make, especially given the high frequency that fireBullet gets called. A better alternative? Change the class field type from Transform to the script class (EnemyBullet, in my case).
Now you avoid the GetComponent call and get a direct reference to the script. This works in my case because the Enemy class only really needs reference to the EnemyBullet script of the particular prefab. It doesn't use the prefab's transform or other components. If you have a case where you're accessing multiple components of a particular prefab, then you can't avoid the calls to GetComponent. However, figure out which component you use the most and set that as the class field type, to reduce the number of GetComponent calls you make.
Tuesday, February 26, 2013
The Unity Way, or....THE UNITY WAY!
When you use a complete "out-of-the-box" game engine like Unity, you're going to be forced into using it in ways that you may not like. Seems obvious, right? You're going to have to do things that may seem counter-intuitive and just deal with the design decisions made by the Unity team. You"ll need to do things "The Unity Way". If you're coming from a different game development environment, regardless of what it is (C++/DirectX/SDL, XNA, etc), there's going to be a great learning curve when starting with Unity.
A good example of this is how you instantiate prefab objects in a scene. Instead of using the new operator to instantiate a new object (or making use of a factory method), you instead must make use of Unity's Instantiate method. Now there is some sound reasoning for this. When you instantiate a prefab, you're actually instantiating all of the multiple components that make up a prefab as well. The Unity documentation explains it well. You can think of the Instantiate method as a somewhat non-traditional factory method that creates and returns your complete prefab instances, so that you don't have to "new" up the code and piece together the prefab by hand.
There is a downside to this. What if you want to change a script parameter of the prefab at instantiation? Hmmmm..., the Instantiate method doesn't support parameter passing, unlike a traditional constructor would. Instead, after you instantiate the prefab, you have to obtain a reference to the attached script, and modify any fields through setter methods.
The above Gist is from my Enemy script. My enemy prefabs travel at a random velocity. Because of this, I need to be able to adjust the speed at which the bullets they fire travel at as well; I was running into scenarios where when the enemy fired a bullet, they both might end up traveling at the same, or near the same velocity. Instead of simply passing speed + BULLET_SPEED_MULTIPLIER as a constructor parameter upon instantiating, I need to do things "The Unity Way".
So I do things "The Unity Way" and I change the fields of my script through setters instead. Not the end of the world. OR IS IT?!?!?! When I went to test my BULLET_SPEED_MULTIPLIER, I noticed that the bullet speeds were unaffected. Why? Because I was setting the enemy bullet speed in the Start method of my EnemyBullet script.
If you read the Start documentation carefully, you'll find out that "Start is called just before any of the Update methods is called the first time". It seems that the execution path is essentially (ignoring several other methods being called in between, I'm sure):
fireBullet -> Instantiate -> setSpeed -> Start -> Update
The lesson of the story is to be careful of what fields you set in your Start method, because that's the value that will be used regardless if you used a setter method immediately after instantiation. It could be argued that this bug would have happened even if I was using a constructor, because Start would have gotten called by the Unity engine later on anyways. I would then counter argue that I probably wouldn't have even used Start to initialize key variables, if I was using a constructor ;)
A good example of this is how you instantiate prefab objects in a scene. Instead of using the new operator to instantiate a new object (or making use of a factory method), you instead must make use of Unity's Instantiate method. Now there is some sound reasoning for this. When you instantiate a prefab, you're actually instantiating all of the multiple components that make up a prefab as well. The Unity documentation explains it well. You can think of the Instantiate method as a somewhat non-traditional factory method that creates and returns your complete prefab instances, so that you don't have to "new" up the code and piece together the prefab by hand.
There is a downside to this. What if you want to change a script parameter of the prefab at instantiation? Hmmmm..., the Instantiate method doesn't support parameter passing, unlike a traditional constructor would. Instead, after you instantiate the prefab, you have to obtain a reference to the attached script, and modify any fields through setter methods.
The above Gist is from my Enemy script. My enemy prefabs travel at a random velocity. Because of this, I need to be able to adjust the speed at which the bullets they fire travel at as well; I was running into scenarios where when the enemy fired a bullet, they both might end up traveling at the same, or near the same velocity. Instead of simply passing speed + BULLET_SPEED_MULTIPLIER as a constructor parameter upon instantiating, I need to do things "The Unity Way".
So I do things "The Unity Way" and I change the fields of my script through setters instead. Not the end of the world. OR IS IT?!?!?! When I went to test my BULLET_SPEED_MULTIPLIER, I noticed that the bullet speeds were unaffected. Why? Because I was setting the enemy bullet speed in the Start method of my EnemyBullet script.
If you read the Start documentation carefully, you'll find out that "Start is called just before any of the Update methods is called the first time". It seems that the execution path is essentially (ignoring several other methods being called in between, I'm sure):
fireBullet -> Instantiate -> setSpeed -> Start -> Update
The lesson of the story is to be careful of what fields you set in your Start method, because that's the value that will be used regardless if you used a setter method immediately after instantiation. It could be argued that this bug would have happened even if I was using a constructor, because Start would have gotten called by the Unity engine later on anyways. I would then counter argue that I probably wouldn't have even used Start to initialize key variables, if I was using a constructor ;)
Saturday, February 23, 2013
Let me get a flash!
I wanted to provide the player feedback upon getting hit. A good way to do this is to "flash" the player, or alternate very quickly between various colors.
There are a few ways you can do this in a 2D game when you're using sprites. However, I'm not using sprites just yet (I'm using the Unity provided basic shape models), so I have to start manipulating the applied materials of the player cube model directly. Simply, on the player object material, you can just interpolate between two (or more, if you're feeling bold) colors rapidly for a short duration.
The following Gist may not be the most elegant, and in fact it's very inefficient, but it's my First Stab Solution™ at the problem. After you're done laughing, be kind and call me out in the comments and show me a better way!
So what do we have? When collision occurs with the player, we begin a InvokeRepeating of the method colorFlash. The colorFlash method will lerp, or interpolate, between the two Colors in the flashColors array. By using the Matf.PingPong method to calculate a lerp time, we add variance to the length of time used for the lerp. The lerp parameter passed into Color.Lerp will be clamped between 0 and 1. If it's 0, the first color is returned, if 1, the second. That's why we divide the result of Mathf.PingPong by flashLerpDuration, to receive a float value between 0 and 1, which allows Color.Lerp to return even more variance in color.
Unfortunately (or fortunately, depending on how you look at it), when you use InvokeRepeating, it'll call the specified method until you make a call to CancelInvoke (I would like for there to be an implementation of InvokeRepeating that accepts a float value as a parameter to specify how many seconds the invoking should last). So, we have to setup a timer, flashInvokeDuration, and when that timer reaches 0, we make the call to CancelInvoke and the player will stop flashing.
There are a few ways you can do this in a 2D game when you're using sprites. However, I'm not using sprites just yet (I'm using the Unity provided basic shape models), so I have to start manipulating the applied materials of the player cube model directly. Simply, on the player object material, you can just interpolate between two (or more, if you're feeling bold) colors rapidly for a short duration.
The following Gist may not be the most elegant, and in fact it's very inefficient, but it's my First Stab Solution™ at the problem. After you're done laughing, be kind and call me out in the comments and show me a better way!
So what do we have? When collision occurs with the player, we begin a InvokeRepeating of the method colorFlash. The colorFlash method will lerp, or interpolate, between the two Colors in the flashColors array. By using the Matf.PingPong method to calculate a lerp time, we add variance to the length of time used for the lerp. The lerp parameter passed into Color.Lerp will be clamped between 0 and 1. If it's 0, the first color is returned, if 1, the second. That's why we divide the result of Mathf.PingPong by flashLerpDuration, to receive a float value between 0 and 1, which allows Color.Lerp to return even more variance in color.
Unfortunately (or fortunately, depending on how you look at it), when you use InvokeRepeating, it'll call the specified method until you make a call to CancelInvoke (I would like for there to be an implementation of InvokeRepeating that accepts a float value as a parameter to specify how many seconds the invoking should last). So, we have to setup a timer, flashInvokeDuration, and when that timer reaches 0, we make the call to CancelInvoke and the player will stop flashing.
Hello GitHub!
So you want to add some source control to your Unity project beyond syncing it to a Dropbox folder? Well it really couldn't be easier, and should be a no brainer for those familiar with Git.
First and foremost, you'll need to change some Editor settings in Unity.
Edit- > Project Settings -> Editor
In the Inspector window, change Version Control Mode to Visible Meta Files. Change Asset Serialization Mode to Force Text.
Then, to create a new Git repository for a Unity project, do the following from a command line.
Note: This is all assuming you have Git installed and added to your path.
cd C:\Path\To\Your\Unity\Project\Root\Folder
git init
git add .
git commit -m "Hello Git!"
Bada bing, bada boom! That's it. I've been going the lazy (A.K.A. more productive) route and using the GitHub GUI to manage my project's local repo and pushing to GitHub. You'll want to setup a README.md and a .gitignore file as well.
My .gitignore file:
[Ll]ibrary/
[Tt]emp/
[Oo]bj/
# Autogenerated VS/MD solution and project files
*.csproj
*.unityproj
*.sln
*.mp3
One word of advice, if you have any files or assets you know you don't want to be included in your Git repo, make your life easier and setup the .gitignore files BEFORE creating your Git repo. Otherwise, it's a real pain to completely remove sensitive data from a git repo and history. The following SO post will show you how to ignore entire directories, if so you please.
First and foremost, you'll need to change some Editor settings in Unity.
Edit- > Project Settings -> Editor
In the Inspector window, change Version Control Mode to Visible Meta Files. Change Asset Serialization Mode to Force Text.
Then, to create a new Git repository for a Unity project, do the following from a command line.
Note: This is all assuming you have Git installed and added to your path.
cd C:\Path\To\Your\Unity\Project\Root\Folder
git init
git add .
git commit -m "Hello Git!"
Bada bing, bada boom! That's it. I've been going the lazy (A.K.A. more productive) route and using the GitHub GUI to manage my project's local repo and pushing to GitHub. You'll want to setup a README.md and a .gitignore file as well.
My .gitignore file:
[Ll]ibrary/
[Tt]emp/
[Oo]bj/
# Autogenerated VS/MD solution and project files
*.csproj
*.unityproj
*.sln
*.mp3
One word of advice, if you have any files or assets you know you don't want to be included in your Git repo, make your life easier and setup the .gitignore files BEFORE creating your Git repo. Otherwise, it's a real pain to completely remove sensitive data from a git repo and history. The following SO post will show you how to ignore entire directories, if so you please.
Wednesday, February 20, 2013
PlayerOutOfBounds
Most likely, you're going to have a player object in your game that you'll want to keep within the boundaries of the screen. When we say "boundaries of the screen", we really mean within the viewport of the camera.
Typically, during your update cycle, you perform a check on the player object current position, and if it's outside of those boundaries, you reset the player's position back within the bounds. The following code snippet is one way of achieving this in Unity, using some arbitrary boundary numbers that represent world coordinates of the bounds of the camera view, that perhaps you happen to figure out while playing and logging your player's position .
This isn't a very robust solution, however. It ignores variances in screen resolution. The world coordinate of 8.0f may happen be the right most bounds of the X-axis at, for arguments sake, 960x600, but not at a higher, or lower resolution. We should also note that the world coordinates will vary based on where you placed your prefabs within the world as well.
To keep things as resolution independent as possible, and thus as platform independent as possible, do the following instead. Note, I call obtainScreenBounds in my player object's Start method. You don't need to call it every Update cycle.
obtainScreenBounds makes use of a key method, ScreenToWorldPoint, "which transforms position from screen space into world space". When you pass in a Vector3 of (0, 0, cameraToPlayerDistance) to this method, a Vector3 that represents the bottom left of the screen, you'll get back a Vector3 representing the same location in world coordinates. When you pass in a Vector3 of (Screen.width, Screen.height, cameraToPlayerDistance), you obtain a Vector3, in world coordinates, that's located at the upper right location of the current screen view. Thus, regardless of what the resolution is, you'll always have the correct screen boundaries to Clamp the player's position with.
Typically, during your update cycle, you perform a check on the player object current position, and if it's outside of those boundaries, you reset the player's position back within the bounds. The following code snippet is one way of achieving this in Unity, using some arbitrary boundary numbers that represent world coordinates of the bounds of the camera view, that perhaps you happen to figure out while playing and logging your player's position .
This isn't a very robust solution, however. It ignores variances in screen resolution. The world coordinate of 8.0f may happen be the right most bounds of the X-axis at, for arguments sake, 960x600, but not at a higher, or lower resolution. We should also note that the world coordinates will vary based on where you placed your prefabs within the world as well.
To keep things as resolution independent as possible, and thus as platform independent as possible, do the following instead. Note, I call obtainScreenBounds in my player object's Start method. You don't need to call it every Update cycle.
obtainScreenBounds makes use of a key method, ScreenToWorldPoint, "which transforms position from screen space into world space". When you pass in a Vector3 of (0, 0, cameraToPlayerDistance) to this method, a Vector3 that represents the bottom left of the screen, you'll get back a Vector3 representing the same location in world coordinates. When you pass in a Vector3 of (Screen.width, Screen.height, cameraToPlayerDistance), you obtain a Vector3, in world coordinates, that's located at the upper right location of the current screen view. Thus, regardless of what the resolution is, you'll always have the correct screen boundaries to Clamp the player's position with.
Tuesday, February 19, 2013
You're alright, kid
I've (finally) been working through the excellent Walker Boys Unity tutorials. First, I want to give a huge thanks for Walker Boys Studios for creating these tutorials. I'm never a fan of video tutorials (I prefer the faster pace of reading along a tutorial), but these are very well done and nicely paced. So on that note, please support their Kickstarter project, Build a Game!
For the most part, I approached these tutorials by getting an idea of what was trying to be accomplished in the particular tutorial video, and just give it a go by myself. Then, I'd watch the video in full and see how the instructor, um, instructed to achieve the same thing.
I'm hosting my Unity games on Google App Engine over at Start Press Games. Once I tinker with the games, adding and improving upon them, to a level beyond recognition of the original intention of the tutorial, I'll be posting them on GitHub for others to view and (hopefully) learn from, but more importantly, for me to receive constructive criticisms as well so I can improve and be called out on my mistakes.
So what are my initial impressions of Unity you say? I know you didn't say anything, you really don't care, but you're getting them anyways!
What I Like
For the most part, I approached these tutorials by getting an idea of what was trying to be accomplished in the particular tutorial video, and just give it a go by myself. Then, I'd watch the video in full and see how the instructor, um, instructed to achieve the same thing.
I'm hosting my Unity games on Google App Engine over at Start Press Games. Once I tinker with the games, adding and improving upon them, to a level beyond recognition of the original intention of the tutorial, I'll be posting them on GitHub for others to view and (hopefully) learn from, but more importantly, for me to receive constructive criticisms as well so I can improve and be called out on my mistakes.
So what are my initial impressions of Unity you say? I know you didn't say anything, you really don't care, but you're getting them anyways!
What I Like
- I'm writing little, to no game engine specific code. Coming from primarily XNA game development, this is nice welcome and a huge boost to productivity. As fun as writing a game engine can be, with my limited time, I'd simply rather not bother. I've yet run into any limitations with the Unity engine itself in terms of what I want to do.
- True multiplatform. With a few clicks, I'm deploying to desktop, web and Android (I have no Mac machine, else I'd deploy to iOS as well!). Outside of having to implement input for touch control for Android, it's been a breeze to target multiple platforms and various resolutions.
- Testing new changes is stupid fast and easy. No need to wait for a full build! Just hit play, and test your changes all within the comforts of the Unity editor. This creates a very nice workflow.
- Asset Store. While I haven't purchased anything yet (I plan to purchase 2D Toolkit this weekend), by browsing the Asset Store, I can see that I'll be able to obtain a proven solution to a problem I (will possibly) face. Is that as bad as copying and pasting code from Stack Overflow? I wouldn't say so, but anything that boosts productivity is a good thing.
- The Unity Editor itself. Unity itself is very nice and intuitive and allows for great workflow.
What I Don't Like
- MonoDevelop. Oh dear God, is MonoDevelop horrible. In fact, the whole scripting IDE situation of Unity is unacceptable in my opinion. Before you judge, do know that a large portion of my job is essentially writing C code in Notepad++ with only Netbeans 3 as a remote debugger (yes, Netbeans 3), so I constantly experience much worse than what Unity offers. I would use Visual Studio 2010, but I really hate that I have to pay $125 to be able to debug in VS2010, and have a decent IDE setup overall (and UnityVS is still missing a lot of necessary features for true productive coding).
- Scripting Reference Documentation. It could be better overall. They do provide some decent code examples, which is nice, but the overall structure and layout needs improvement (how method signatures are conveyed, for one). A more minor nitpick, but there's no option to default the script examples to C#, over the current default of JavaScript.
- Reflection to determine method implementations. Instead of allowing developers to actually override methods, reflection is used at creation to identify all of the (what should have been traditionally virtual) methods of the Unity engine that the developer implements (such as Update). This was done to optimize performance (only call the methods that are actually implemented), but makes identifying override methods more difficult. The bigger issue is that you can't use IntelliSense to implement the method, and thus need to make sure you have that trusty Scripting Reference web site open and by your side at all times to get the method signatures correct.
- Public variables exposed in designer view. Now, I'm sure many people would expect this as an item on my "What I Like List", and I understand that this feature is so that non-coder designers can change variables with ease, and without looking at code, but since I am a coder, I instinctively change the values within code. However, if the value was ever changed within the editor, changing the value in code is ignored, thus a bug. A lazy solution here would be to never declare public variables.
Sunday, February 3, 2013
Translate that for me, will you?
To move an object, specifically the player, you have to translate that object along the X, Y and Z axis. The three basic steps in calculating the translation value are:
In Unity, there are several ways to obtain the input value range of the player. You could poll for keyboard or other input events and translate the object along the particular axis that corresponds with the key pressed. For example, in a 2D game where the origin is in the bottom left of the screen (positive Y moves up the screen, positive X moves to the right of the screen),
float playerSpeed = 10.0f;
if (Input.GetKey(KeyCode.W))
{
transform.Translate(playerSpeed * Time.deltaTime, 0, 0);
}
if (Input.GetKey(KeyCode.S))
{
transform.Translate(-playerSpeed * Time.deltaTime, 0, 0);
}
// etc...
This is a very verbose method of getting the player input and translating the player object. A better, more preferred method is to use GetAxis.
float playerSpeed = 10.0f;
float xTranslation = Input.GetAxis(“Horizontal”) * playerSpeed * Time.deltaTime;
float yTranslation = Input.GetAxis(“Vertical”) * playerSpeed * Time.deltaTime;
transform.Translate(xTranslation, yTranslation, 0);
This is a much more concise method. Through Input.GetAxis, we pass in the name of the virtual axis, as setup in the Input Manager, and get returned the input value range (-1 to 1). Horizontal and Vertical are default input settings, which make use of WASD, arrows, and a gamepad left analog joystick.
If the player is holding down the ‘D’ key, Input.GetAxis(“Horizontal”) will return a value of 1. If the player is holding down the ‘S’ key, Input.GetAxis(“Horizontal”) will return a value of -1. Multiplying that by playerSpeed will cause the player game object to move along the X axis in a positive direction by 10 units. Further multiplying this by Time.deltaTime allows us to move independent of the computer speed, so we get same rate of movement regardless of how fast the player’s machine may be.
- Obtain the input value range
- Multiply by the desired speed
- Multiply by delta time (to allow the object to move based on time and not computer speed)
In Unity, there are several ways to obtain the input value range of the player. You could poll for keyboard or other input events and translate the object along the particular axis that corresponds with the key pressed. For example, in a 2D game where the origin is in the bottom left of the screen (positive Y moves up the screen, positive X moves to the right of the screen),
float playerSpeed = 10.0f;
if (Input.GetKey(KeyCode.W))
{
transform.Translate(playerSpeed * Time.deltaTime, 0, 0);
}
if (Input.GetKey(KeyCode.S))
{
transform.Translate(-playerSpeed * Time.deltaTime, 0, 0);
}
// etc...
This is a very verbose method of getting the player input and translating the player object. A better, more preferred method is to use GetAxis.
float playerSpeed = 10.0f;
float xTranslation = Input.GetAxis(“Horizontal”) * playerSpeed * Time.deltaTime;
float yTranslation = Input.GetAxis(“Vertical”) * playerSpeed * Time.deltaTime;
transform.Translate(xTranslation, yTranslation, 0);
This is a much more concise method. Through Input.GetAxis, we pass in the name of the virtual axis, as setup in the Input Manager, and get returned the input value range (-1 to 1). Horizontal and Vertical are default input settings, which make use of WASD, arrows, and a gamepad left analog joystick.
If the player is holding down the ‘D’ key, Input.GetAxis(“Horizontal”) will return a value of 1. If the player is holding down the ‘S’ key, Input.GetAxis(“Horizontal”) will return a value of -1. Multiplying that by playerSpeed will cause the player game object to move along the X axis in a positive direction by 10 units. Further multiplying this by Time.deltaTime allows us to move independent of the computer speed, so we get same rate of movement regardless of how fast the player’s machine may be.
Monkey In The Middle
I wrote this a few years ago when I was working on a MMO Scrabble game using Google Web Toolkit and Google App Engine. This chronicles the decision the team made for choosing a NoSQL database solution such as GAE. I thought I would add it here for safe keeping. Enjoy!
When considering walking down the NoSQL path, the needs of your application will need to dictate, the two priorities that you decide to go with in your NoSQL, distributed database solution. Choosing the correct combination is absolutely critical and choosing the wrong combination could very easily have a dire impact on your business. For example, Amazon claims that just an extra one tenth of a second on their response times will cost them 1% in sales. This should really drive home the importance of correctly choosing your applications database needs. No pressure ;)
There’s simply no “best” database solution to any application. It’s a constant battle of tipping the scales. It’s almost guaranteed that regardless of what solution you go with, it’ll have a negative impact somewhere. The trick is to limit that negative impact or at least direct it towards a aspect that can “afford” to be hit by it, where perhaps the developers can “make-up” for the short coming within the application’s design and implementation.
For WordWars, we knew from the beginning we wanted a database solution that was easily scalable, being able to handle 30 users one day, and scale up to 10,000 users the next day (we could only be so lucky to see a large jump of users like that!), with virtually no mediation required. With an increased user base, we also want to maintain the same high level of performance as our user-base fluctuates, while keeping in mind the fact that we have little to no money to scale vertically. We also need to rely heavily on transactions, due to the nature of gameplay and high probability of having many concurrent users playing at the same time, attempting to access the same data. We need consistency so that every user viewing a single board sees the same thing. We also need a certain level of data integrity. For example, we simply need to be able to gracefully handle the use case of two users attempting to play a word on the same board location, at the same time.
Google’s Big Table database solution attempts to play the middle ground. It is NoSQL based, offering a high level of speed and scalability that relational database tend to fail to deliver, but also provides transactions, a common feature standard in relational databases, but often missing from NoSQL databases. Big Table provides full transaction support, using optimistic concurrency. As you can see from the CAP theorem diagram above, Big Table provides Consistency and Partition Tolerance, which satisfies the needs of WordWars nicely. Yes, we are giving up complex SQL reporting. However, Big Table does use a similar query language called GQL, but at this time it’s still very simple and infant in it’s capabilities, which is more then enough to satisfy what we predict to be our reporting needs.
We are also sacrificing Availability, which may appear to be a huge sacrifice given the type of application we are developing. A failed transaction will happen when using Google’s datastore, for various reasons (timeout, concurrent modification issues, etc). It’s inevitable, it’s the nature of the beast. However, we are developing our application to handle these failures gracefully and greatly limit the impact of any transaction failure. It’s a sacrifice we can make-up for in our application design. If we decided to choose Availability over Partition Tolerance, then we would be losing the latter property completely, with virtually no way of recovering it or making up for it in some other way, at least not easily or efficiently.
Through Google’s App Engine service, we could start using Big Table immediately, have a full NoSQL database at our disposal within minutes with virtually zero effort on our part, leaving Google to the administration duties. Most importantly, we get everything completely free. We will only start getting charged once we have regular users and furthermore once we surpass the free quota thresholds. Since we are in the very early stages of development and prototyping, this aspect alone is enough to drive us toward Google App Engine.
Choosing a database solution for any application is an extremely difficult, but critical decision. You are deciding the backbone of your application, whats going to be used to capture and store your applications precious data. When viewing all of the options out there, you need to evaluate their strengths and weakness, whether they play into or against the needs of your application. You need to look at your application and attempt to envision the end product. How many users will be using it? Will these users be accessing your data concurrently? Will you have a big fluctuation in the number of concurrent users at a time, say 50,000 users between the hours of 5 -8pm, but only 1,000 users during 6am-11am? How will you handle a large jump of overall users due to the applications increased popularity (hopefully!). In terms of hard disk space, how much data do you expect to store? At what rate will this grow? How complex do you expect your schema to be? Who do you have that can be your database administrator, and what kind of technologies can they manage and administrate based on their current skill set? What are the costs of operating and maintaining the server that will be housing the database? These are just a sample of questions that need to be answered.
Relational databases easily dominate the market. They are the popular choice, and for good reason. They have been proven to be viable solutions for a vast range of applications, from online retail stores to complex AAA MMORPGs. They have great transaction support, providing full ACID qualities. They can handle a high number of users concurrently accessing the same data. They support highly complex SQL queries and data manipulation, allowing developers, designers and business executives to query against their data to build much needed data metrics and analysis, able to answer questions such as “on our big annual 4th of July sale, for years 2003-current, for which item which was available during that date range, was the third best selling in our New York City customer base”, using these reports to further drive their business and sale tactics, or identify inefficiencies and help improve overall sales.
However, relational databases are not perfect. They have their weaknesses, more specifically in the realm of performance, scalability and object-relational impedance mismatch issues. Yet most applications simply choose to work around these imperfections due to a simple case of “the benefits of relational databases greatly outweigh their negative impact on my application”. In the case of massively-multiplayer-online-games (MMOGs), performance, scalability, completely avoiding object-relational impedance mismatch, high quality transaction support and complex SQL reporting are all needed, practically equally so. It’s been said that with regardless of which database solution you choose, you are allowed to only pick two items from the latter list and apparently most developers need highly complex SQL reporting as number one, leaving the second choice up in the air. Historically, however, transaction support tends to be a very close second in terms of desired features.
Cryptic Studios, in the quest for coming up with the perfect database solution, have gone as far as designing and developing their own database management systems, CrypticDB. The CCP Team of EVE Online actually use a single-shard SQL server architecture, and to combat performance issues, have thrown large sums of money into a mind bogglingly beefy server hardware configuration, A.K.A. scaling vertically (more on that later). While throwing money at the problem may be a viable solution to some, for many (start-ups for example) that’s not an option, at least it’s not the best.
From an engineers perspective, relational databases introduce the problem of object-relational impedance mismatch, making it difficult to translate the data that is used in a object-oriented application to store nicely within the relational databases strict structural confinements and incompatible datatypes. In order to store complex data structures, we are forced to develop wrapper classes to do the translation, which if not done perfectly, will have severe impact on performance, possibly so severe that the end user may be affected, like being forced to wait two seconds to loot a green item from a dead zombie. The issue of object-relational impedance mismatch is an extensive one, beyond the discussion of this particular post.
Luckily, we aren’t stuck with just relational database management systems to choose from. Leaving the realm of relational databases introduces the broad range of NoSQL databases. NoSQL are object-oriented based, and in general have the SQL query language omitted (thus the name, NoSQL). There are many different varieties of NoSQL databases, being categorized by their data storage methods. NoSQL databases were developed to primarily solve the issue of scalability, to scale horizontally rather than vertically (CCP, if you remember, decided to scale their database vertically by increasing their hardware configurations). Scaling horizontally means adding more nodes to a system, creating a distributed network. In a distributed environment, data integrity becomes an immediate concern, so the ultimatum of choosing “consistency vs. availability” comes into hand.
NoSQL was also introduced to deal with the issue of object-relational impedance mismatch, being able to handle and store a much wider range of datatypes and structure than a RDBMS could. As an extension, NoSQL also allows handling large volumes of data, quickly retrieving such information, virtually eliminating the need for expensive JOIN operations (which most NoSQL solutions decline to even offer). In essence, NoSQL offers either a high level of data integrity or availability and better compatibility and support for complex and object-oriented data.
When talking NoSQL, we can’t neglect to talk about the CAP theorem, first proposed by Eric Brewer in 2000. The CAP theorem states, in a distributed computing environment, you can optimize for only two of three priorities, Consistency, Availability and Partition Tolerance.
However, relational databases are not perfect. They have their weaknesses, more specifically in the realm of performance, scalability and object-relational impedance mismatch issues. Yet most applications simply choose to work around these imperfections due to a simple case of “the benefits of relational databases greatly outweigh their negative impact on my application”. In the case of massively-multiplayer-online-games (MMOGs), performance, scalability, completely avoiding object-relational impedance mismatch, high quality transaction support and complex SQL reporting are all needed, practically equally so. It’s been said that with regardless of which database solution you choose, you are allowed to only pick two items from the latter list and apparently most developers need highly complex SQL reporting as number one, leaving the second choice up in the air. Historically, however, transaction support tends to be a very close second in terms of desired features.
Cryptic Studios, in the quest for coming up with the perfect database solution, have gone as far as designing and developing their own database management systems, CrypticDB. The CCP Team of EVE Online actually use a single-shard SQL server architecture, and to combat performance issues, have thrown large sums of money into a mind bogglingly beefy server hardware configuration, A.K.A. scaling vertically (more on that later). While throwing money at the problem may be a viable solution to some, for many (start-ups for example) that’s not an option, at least it’s not the best.
From an engineers perspective, relational databases introduce the problem of object-relational impedance mismatch, making it difficult to translate the data that is used in a object-oriented application to store nicely within the relational databases strict structural confinements and incompatible datatypes. In order to store complex data structures, we are forced to develop wrapper classes to do the translation, which if not done perfectly, will have severe impact on performance, possibly so severe that the end user may be affected, like being forced to wait two seconds to loot a green item from a dead zombie. The issue of object-relational impedance mismatch is an extensive one, beyond the discussion of this particular post.
Luckily, we aren’t stuck with just relational database management systems to choose from. Leaving the realm of relational databases introduces the broad range of NoSQL databases. NoSQL are object-oriented based, and in general have the SQL query language omitted (thus the name, NoSQL). There are many different varieties of NoSQL databases, being categorized by their data storage methods. NoSQL databases were developed to primarily solve the issue of scalability, to scale horizontally rather than vertically (CCP, if you remember, decided to scale their database vertically by increasing their hardware configurations). Scaling horizontally means adding more nodes to a system, creating a distributed network. In a distributed environment, data integrity becomes an immediate concern, so the ultimatum of choosing “consistency vs. availability” comes into hand.
NoSQL was also introduced to deal with the issue of object-relational impedance mismatch, being able to handle and store a much wider range of datatypes and structure than a RDBMS could. As an extension, NoSQL also allows handling large volumes of data, quickly retrieving such information, virtually eliminating the need for expensive JOIN operations (which most NoSQL solutions decline to even offer). In essence, NoSQL offers either a high level of data integrity or availability and better compatibility and support for complex and object-oriented data.
When talking NoSQL, we can’t neglect to talk about the CAP theorem, first proposed by Eric Brewer in 2000. The CAP theorem states, in a distributed computing environment, you can optimize for only two of three priorities, Consistency, Availability and Partition Tolerance.
Image Source: http://blog.nahurst.com/visual-guide-to-nosql-systems
There’s simply no “best” database solution to any application. It’s a constant battle of tipping the scales. It’s almost guaranteed that regardless of what solution you go with, it’ll have a negative impact somewhere. The trick is to limit that negative impact or at least direct it towards a aspect that can “afford” to be hit by it, where perhaps the developers can “make-up” for the short coming within the application’s design and implementation.
For WordWars, we knew from the beginning we wanted a database solution that was easily scalable, being able to handle 30 users one day, and scale up to 10,000 users the next day (we could only be so lucky to see a large jump of users like that!), with virtually no mediation required. With an increased user base, we also want to maintain the same high level of performance as our user-base fluctuates, while keeping in mind the fact that we have little to no money to scale vertically. We also need to rely heavily on transactions, due to the nature of gameplay and high probability of having many concurrent users playing at the same time, attempting to access the same data. We need consistency so that every user viewing a single board sees the same thing. We also need a certain level of data integrity. For example, we simply need to be able to gracefully handle the use case of two users attempting to play a word on the same board location, at the same time.
Google’s Big Table database solution attempts to play the middle ground. It is NoSQL based, offering a high level of speed and scalability that relational database tend to fail to deliver, but also provides transactions, a common feature standard in relational databases, but often missing from NoSQL databases. Big Table provides full transaction support, using optimistic concurrency. As you can see from the CAP theorem diagram above, Big Table provides Consistency and Partition Tolerance, which satisfies the needs of WordWars nicely. Yes, we are giving up complex SQL reporting. However, Big Table does use a similar query language called GQL, but at this time it’s still very simple and infant in it’s capabilities, which is more then enough to satisfy what we predict to be our reporting needs.
We are also sacrificing Availability, which may appear to be a huge sacrifice given the type of application we are developing. A failed transaction will happen when using Google’s datastore, for various reasons (timeout, concurrent modification issues, etc). It’s inevitable, it’s the nature of the beast. However, we are developing our application to handle these failures gracefully and greatly limit the impact of any transaction failure. It’s a sacrifice we can make-up for in our application design. If we decided to choose Availability over Partition Tolerance, then we would be losing the latter property completely, with virtually no way of recovering it or making up for it in some other way, at least not easily or efficiently.
Through Google’s App Engine service, we could start using Big Table immediately, have a full NoSQL database at our disposal within minutes with virtually zero effort on our part, leaving Google to the administration duties. Most importantly, we get everything completely free. We will only start getting charged once we have regular users and furthermore once we surpass the free quota thresholds. Since we are in the very early stages of development and prototyping, this aspect alone is enough to drive us toward Google App Engine.
Subscribe to:
Posts (Atom)