QubeKwest Blog

For the first time in months I have actually done a bit of development on QubeKwest.  It’s a new year and I’m hoping this one sees a lot of progress on the game.  Comically, I hit my limit on development laziness before the new year, so I can’t even claim that this is some sort of new year’s resolution.  The reasons for my resuming development aren’t exactly what you might expect.

The first source of the “itch” to start coding again came from playing Minecraft with my brother again.  One day he randomly called me up and told me he wanted to play again.  I accepted his offer and have been playing for a couple of months with him.  This inspired the same sorts of thoughts that caused me to start writing QubeKwest in the first place.  In other words, playing Minecraft simply made me want to make something like Minecraft only better.

Part of the reason for stopping the development for a while was that the unimaginable scale of the project had rather started to soak in.  That in turn made me feel like the progress I was making was painfully slow because after so long, and so much work, I still essentially have nothing to show for my efforts.

The second motivation to resume and what helped to remove a source of not feeling up to the task of writing QubeKwest actually came from a book I’m reading.  I’m reading my way through Console Wars: Sega, Nintendo, and the Battle that Defined a Generation (which I’m enjoying immensely) and at one point they were talking about how long it took to develop “The Legend of Zelda: A Link to the Past.”  This SNES game from 1991 lived entirely in a 1MB cartridge and took something like 3 years to create for a whole team of video game writing experts.  More impressively, and the part that really hit home for me, was the fact that the game took 60,000 hours of development time.

So Zelda took that long to make for a group of coding wizards.  A game that had very few of the problems to solve that QubeKwest has (though I’m sure it had many others that QubeKwest doesn’t).  There were no clients or servers or networking.  There was never more than one player playing at a time.  The world was expertly hard coded by level designers instead of a completely computer generated world that is dynamically modifiable by any number of players simultaneously.

To be clear, I’m not putting down the efforts of the game creators at Nintento and that particular Zelda game is one of my favorite games of all time.  I’m simply pointing out that it felt like a bit of justification that a game I’m writing from scratch, by myself, engine and everything, is taking so long to make.  When I read that I smiled and honestly felt better about the progress I’d made so far.

Now in a situation where I wasn’t feeling down on myself for a lack of visible progress, and motivated to make a better Minecraft-type game, I started working again.  Here’s hoping that a new year brings with it an incredible amount of progress and continued motivation.  Happy new year everyone.

As a bit of a diversion from all the networking code I’ve been wrestling with over the past several months, I decided it was high time I started thinking about how the electronics would operate in my game.  I realize of course that thinking about how the game itself would work, or look (graphically) might be an even better use of my time, but when you have the motivation to try to figure something out you should jump on it.

When you consider how things work in Minecraft, you often think of their somewhat miraculous redstone system.  Redstone can be used to make things move and light up, and the true experts have even built simple computers using vast collections of redstone pieces.  It is easily one of the most interesting parts of Minecraft to me, but when playing the game I often feel its limitations and occasionally odd requirements take away from my pleasure.

For example, I built a “monster farm”  in Minecraft that provides a dark place for monsters to spawn, and then it attempts to kill them with an elaborate collection of pistons and flowing water.  The circuit involved had big loops of repeaters to create delays and a couple of XOR gates to drive decisions.  The XOR gates I found the diagram for online (because I couldn’t figure out how to do it on my own) required amusingly X-shaped structures that used  something like 5 redstone torches, several blocks that were carefully spaced apart, and a few bits of redstone wire.  I found this highly inefficient.

This is what drove me to the concept that QubeKwest should simply have electricity.  You know, good old fashioned power sources spitting out power that works its way through a circuit to ground.  Once I’d made that choice, it seemed like it would be pretty straight forward to simply provide several of the things that were tricky to build in Minecraft as blocks.  If you want an XOR gate, make an XOR gate block and drop it on the ground.  That’s it.

QubeKwest now has two new packages related to the logical underpinnings of the electronics system in the game.  The base com.qubekwest.electric package and the nested com.qubekwest.electric.component package.  These were separated because I knew going into this that there would be dozens of components and I wanted a separate place to put them all that wouldn’t end up confusing the basic things.

Once the structure was in place, I spent something like 3 days pondering how things would work.  Then I spent 2 more days drawing things in my notebook related to electric stuff.  Then I started coding.  Got it utterly wrong.  Refactored it.  Still wrong.  Massive changes made things much better, but still not great.  Made more progress, realized my current layout was making things unnecessarily inefficient.  Came up with another change to fix that.  Spent another day refactoring everything.  Then I had some things working for real, and some of my tests written to make sure they were working.  Several days later, I’m starting to really have something here.

The basic idea is a collection of components that have at most 4 inputs or outputs.  After all, everything is 4-sided in a world made up of cubes.  Keeping in mind that all of the components are logical constructs only, here is the list of components I’ve written so far and some brief descriptions of what they are:

• Power Flow
  • Power Source – Any source of power.  In the game this could mean a battery, nuclear reactor, solar panel, or anything else that puts out power.
  • Ground – The final target of all circuits.  Without getting to ground, the circuit won’t do anything.
• Light Sources
  • LED – A light source that only lights up when powered.  These only have 1 brightness level.
  • RGB LED – A single LED that provides separate levels of brightness for red, green, and blue parts.
• Logic Gates – Power level 0 = false, 1-15 = true.
  • And – Output is true only if both inputs are true.
  • Nand – Output is true unless both inputs are true.
  • Or – Output is true if either input is true or both inputs are true.
  • Nor – Output is true only when both inputs are false.
  • Xor – Output is true if one or the other input is true, but not both.
  • Xnor – Output is true only if both inputs are the same.
  • Not – Output is the opposite of the input.
• Resistance
  • Resistor – Reduces the power level by a user set amount.
  • Potentiometer – Reduces the power level by an electrically controlled amount.
• Timing
  • Timer – Takes in constant power and outputs it on a user set schedule.
  • Variable Timer – Takes in constant power and outputs it on an electrically controlled schedule.
• Switching – These switches could take several forms each in the game.
  • Switch – Normal on/off switch activated by the user.
  • 3-Way Switch – Never truly off, just sends input power to either one output or the other.
  • 4-way Switch – Never truly off, either passes power straight through, or flips the conductors to be crossed.
  • Selector Switch – Like a 3-way switch, but with 3 outputs.
  • Dual Switch – Single control, dual switch.  This switch has 2 independently wired switches in it that share a single on/off state.
  • Separate Flow Switch – Has 2 paths for power though it, and either allows one path to be closed, or the other.
  • Relay – A normal switch except that it is controlled electronically instead of by the user.
  • Lit Switch – A normal switch that has a separately wired light in it.  The state of the switch is not connected to the state of the light.
• Wiring
  • Wire – A simple conductor.
  • Wire Bundle – A bundle of 9 independent conductors.
  • Wire Junction – A junction point between up to 4 wires.
• Coming soon…
  • Multiplexer – A device that takes in bit input lines, and outputs a power level that is the conversion of those inputs to a number.
  • Demultiplexer – A device that takes in a single power level number and enables separate power lines that represent the bits in that level.
  • Wire Bundle Breakout – A device that gives separate access to each conductor in a wire bundle.

The above work represents something like 5000 lines of code added over the last week.  This diversion has proven to be quite entertaining and productive indeed, and until I’ve finished testing the various components I’ve created, I can’t really say I’m done.

It’s been a long hard road with the networking engine, and I’m absolutely no where near the end…  but I have successfully pinged my server from my client!

Proof, because pictures make everything better, click it to see the animation.

There are loads of problems still, and a lot of those are simply because I haven’t even coded some of the bits that are needed to make things work correctly.  I’m very happy to see a client-server round trip working, and it gives me a lot of hope that this version of the network code will actually be the one I use without having to start over again.

Here is a list of some of the things I know don’t work correctly yet:

• When a client disconnects from the server, there seem to be some weirdness around the server accepting new connections afterwards.  (Not sure if different ways of disconnecting cause different issues.)
• When multiple clients are connected to the server, there is no way for the server to accurately identify which one is which, so things that are expected to return to the client that sent them (like a ping) actually end up going to the client that connected first.  (This one is easy, the clients don’t yet have the ability to register themselves and get their “source ID” from the server.  That means they all use the same hard coded ID.)
• When the server sends a message that is intended as a broadcast, it is not actually broadcasted to all connected clients.  (I think this is the same problem as the one above, but I listed it separately because I am not positive yet.)
• There is no way yet to do directed messages from either the client or the server.  (There isn’t a way for clients to request the list of valid target IDs and their associated names yet.)

This list probably isn’t complete, but it does represent some of the more obvious things that still need to be added to the network engine.  Please don’t take this as negative, I’m thrilled to have successfully gotten a round trip ping to work, and I’m very happy that the list above no longer includes things like “make the network engine work at all.”  I call this a total victory with a little bit of a TODO list.  :D

So as usual around here lately, I’ve been working a lot on the network code.  I suspect if you read this blog with any amount of consistency, or if you are reading all of the posts all at once as a form of weird entertainment, you are probably getting a bit tired of hearing about it.  I am too, so I get it.  The problem is that networking is hard and complex and easy to screw up.  The other problem is that it’s such an important core piece of QubeKwest, that I have to get it right to have any hope of building a game on top of it.

Today’s networking story is about network events.  These magical things fly around inside the various parts of the networking engine and change flags on selection keys and allow it to behave accordingly as it does its thing.  The start of the problem was that I noticed that every single time I connected to the server from the client, the client took almost exactly 3 seconds to do so.  To be fair, I could ignore this problem and say “well it’s slow, but it works,” at least for a little while.  That didn’t feel right to me, so I started to dig in and study the code to try to figure out both why it was happening, and why it was so consistent.

Thanks to 3 seconds being the selector’s select() method timeout, and a time that I chose when I realized that the network engine was never shutting down properly, I had a place to start in my search.  Unlike so many other issues with the networking code, this one jumped out immediately.

The process of connecting to the server from the client is a two step process.  The first step is to set up some stuff and schedule the second part to occur once the selector gets to it.  The second part is for the selector to realize it has a connection to complete, and to, well, complete it.  On the surface, this all seems pretty straight forward, but these things are taking place in two different threads.  The client network engine is up and spinning in its own thread, and the UI has its own thread too, and that will be where the connection process is started.

If we look at the client network engine thread first, we see that it does something a bit like this:  process any events that came in, select() on our selector for things that are ready, and then process the stuff that’s ready.  Then it loops back to the start of that and does it all over again.  The problem is that select() is a blocking call.  That means that in a network that isn’t busy doing anything (like before it has connected) there is nothing to cause the select() to ever finish waiting for something to do.  This is the exact spot that I added the 3 second timeout, so it would have the chance to check if the loop is supposed to end because the engine is being shut down.

Do you see the problem?  My selector is sitting there waiting for something to have its flags changed by an event.  The step where events change things happens somewhere else in the loop that isn’t happening because select() is blocking the loop from running.  This causes network events to be effectively ignored for 3 seconds.  That means the pattern is really more like this: when select() times out in 3 seconds, it immediately loops back around, processes the events, tries to select() again, and that ends immediately since there is a flag set from the event.  Just like that, the second part of the connection process immediately processes, but after the 3 second timeout.

Now comes the hard part.  Determining what was happening was easy, coming up with a way to fix it will be quite tricky and I believe to have any shot at it working, it will need to be yet another thread.  This new one has only one job to do.  Watch for events and process them immediately in a thread that isn’t being blocked by the select() method.  I don’t think the code involved in the new part will be terribly difficult to write, but it will require a bit of messing with the client and server engine codebases to integrate the proper use of it.