Go to Gallery           Register for MyTrainz | Login to MyTrainz

How full is your Trainz glass?

By Tony_Hilliam, February 02 2018

Trainz has been around for over 15 years and there's a good reason for that - our passionate fans have been returning year after year and extolling the many positives of Trainz, and new customers have been coming along to join in the fun.

Underlying all those positives, there has always been a recurring negative theme of "why isn't Trainz even better?" and "why are there still bugs?". 

Is this just a case of people being "glass half full" optimists or "glass half empty" pessimists? We decided to take a look at answering some of those "glass half empty" questions...

Q. When will the next update for TANE be released?

A. Such a good question to start with :-). We'd love to provide a date right now. Everyone would be happy knowing that the next update was X days or weeks away. But then there's a good chance that date could be missed. Perhaps we haven't estimated the times for the known list of tasks correctly. Perhaps there will be some new unknown tasks added to the list. Perhaps there will be a hardware failure where we spend two weeks reconfiguring the entire development platform (that one has actually happened!).

Alternatively, we could announce a release date, then stick to that schedule. Choosing this option could potentially mean the product is released before it's ready. We may not have completed all the code, tweaked the necessary artwork, or, especially in the case of Trainz, tested all the potential variables found on end-user systems rather than our Internal QA test case systems.

We'd like to thing we've learned our lessons about making announcements and then letting our customers down by announcing delays. Our policy now, more than ever before, is to keep developing, updating, tweaking, fixing and testing, and release when we're confident the product is ready.

Glass half full person says "release when it's ready."
Glass half empty person says "stick to the schedule and make sure there are no problems!"

Q. Why then can new bugs appear, even after rigoruos testing is complete?

A. Yes, it's true - even after many weeks of testing, it's still possible for bugs to slip through the net. For this we are truly sorry - but, we're not alone with this problem. Even much larger companies with hundreds (or thousands) of developers, many thousands of testers and millions of customers, fail to produce bug-free software. Developing software is a tough job and developing a software platform that can be expanded upon by thousands of end-users is even tougher!

Even when you think everything is fine, there's a chance that something gets overlooked. Here's a classic example from the latest release:

One of the bugs reported post-release had been present for well over a month and no-one noticed. That's with more than 1,000 people having the beta version installed, and the bug was quite visible (especially once it was pointed out). The reason for the bug? There was a small change to some skybox rendering code, which in turn meant that at a certain draw distance, any terrain higher than 500m would fall behind the skybox and appear invisible. So the tops of mountains were chopped off. For this to be seen, of course you had to be in a map with mountains higher than 500m, be looking in the right direction and know what the scene was meant to look like previously.

The reason no-one noticed? Perhaps they did, but didn't let us know. Perhaps they were looking at all the other things it is possible to look at and missed that one. Perhaps we just need to broaden our test base even further.

Glass half full = there are fewer bugs now than there were before.
Glass half empty = these new bugs should not have been allowed to slip through.

Q. How did the latest AI issues appears in SP2 HF1?

A. Ah yes - another great question! This is an interesting case. Firstly, the AI still does a great job for most users (most of the time). Command the driver to go from A to B - in most cases it works. However, in certani circumstances, this go wrong.

Here's a short synopsis of how this issue arose:

Late in the last development cycle, there were a couple of bugs being reported about AI trains getting blocked at signals and stalling without notifying the player. We decided these bugs were high enough priority to fix in this cycle, and so we fixed the bugs. We tested the new code against the original test cases and sure enough, the AI now worked as expected - the trains no longer got stuck at the signals or stalled unexpectedly (and if they were genuinely stuck, they would now report the issue).

Further on in our development cycle, with bug reports at a trickle, and internal testing complete, we determined that there were no major problems with the build that would prevent us releasing it. So the update was announced, and the general public version released.

A few weeks after release, reports started coming in about AI issues. As we gathered information, it was evident that not everyone was affected but for some it was a big headache. As we gathered more information, we discovered a range of issues that, in certain combinations, would cause problems for the AI. These variables included camera position, dual or single cab locos, the number of AI drivers in the route, the type of signals used, the rules used, which version of Trainz the trackmarks were placed in, and even the frame rate at certain times. Note that not all these things were required for each issue to appear.

This was almost a "perfect storm" of events, and one that lay dormant just long enough to remain hidden off our list of "known issues" beyond our release date.

As a result of feedback from a range of Trainz users, we've put together a set of more detailed test cases that include all of these variables. We're currently working to ensure that the next update resolves these problems and makes the AI run smoother that ever before.

Glass half full = this is a temporary setback.
Glass half empty = this spoils the experience.

Q. Why can't you release smaller updates more often?

A. We've looked at this option and we'd love to move towards more frequent releases. Microsoft with their "Insider Program" currently have 3 release cycles:

  • "Fast" (which receives new builds as they are released)
  • "Slow" (which receives new builds on a delay after it is deployed to Fast ring users), and
  • "Release Preview" which goes to the remaining Insders, but not the "General Public".

We aim to introduce a similar program as well.

For the final release version of TANE, it's worth noting that there are 10 different versions to prepare - different platforms, different online stores, different content sets, different rights management. Each one needs to be built and tested. There need to be patches (online and offline), and we potentially need to create updates for other systems such as the DLS, DLC etc. Under the current process, it takes about a week to release a single update so we can't be doing that every week!

Also, for maximum efficiency on the development side, we try to maintain only one codebase. So before any new official update, we need to integrate and stabilise everything, then begin external testing, then release to the public. Our goal is to introduce these additional layers of testing into our next product cycle.

Glass half full = I like the idea of smaller, more frequent updates for future versions
Glass half empty = why coudn't this be happening for the current version?

Q. Why can't you fix all the bugs first before releasing the next version of Trainz?

A. As we've said earlier, why can't any company fix all the bugs? The answer of course is that it just doesn't happen.

That's not to say we're not fixing bugs. We're fixing bugs every day! But, there are quite a few of them and, in fact, we have over 1,000 items on our "To Do" list right now. Most of them are low priority, but to some people, that one bug is the only thing they would like to see fixed right now. We hope we can make you all happy one day :-).

Glass half full = keep working on improving Trainz.
Glass half empty = every new release should be perfect out of the box.

Q.. Why can't the beta testers find the bugs?

A. One of the main reasons bugs exist is the sheer scope of what Trainz can do. Every volunteer beta tester who downloads and installs their test version of Trainz wants to help the community by finding and reporting problems. Generally they test against their "regular Trainz content and procedures". This process finds a number of issues, and this is why you will see hundreds of reported fixes in each update we release.

However (we do say that a lot), testing every potential combination is impossible. Hundreds of thousands of assets exist for Trainz. Those assets have been made for a dozen different version of Trainz over the past 15 or so years. Many have the potential to have scripted behaviour that interacts with other elements of a "world". Throw into the mix a huge variation in operating procedures from running single trains in cab mode to hundreds of trains in AI mode, or simply making worlds in Surveyor and testing them out, and you can see that ultimately we need every end user to also be a bug-reporter so that their individual circumstance can be addressed (at some point, based upon priorities).

Glass half full = beta testers find and report loads of bugs.
Glass half empty = beta testers should find every bug.

Q. Why do you keep on adding new things before fixing existing issues?

A. If we focused on solely fixing bugs, Trainz would still look like this:

...instead of like this:

(Click the image above and drag the slider to see TANE on the left and Trainz Next on the right).

If we didn't keep adding new features, functions and graphics capabilities, we wouldn't sell any new copies of Trainz (or any DLC). This would mean we couldn't pay our staff.

Our staff need income to survive, and customers who buy new products provide that income. No staff means no DLS, no support, no bug fixes, and no future for Trainz. And that isn't something anyone wants to see!

Glass half full = I love the new features and new graphics.
Glass half empty = I want the new features and graphics and all the bugs fixed too.

Q. Why does it all take so long?

A. Yup - we wish it was quicker than it is too. Trainz is a niche product with enormous scope and a lot of flexibility and we are a small development team. That combination means we can't do everything as quickly as we'd like, but we do our best. Just as we'd like Christmas to come round more than once a year (and perhaps birthdays less often), we have to make do with what we have :-).

Glass half full = keep working as hard as you can to make Trainz better.
Glass half empty = work harder and faster.

Q. When will Trainz Next be named/announced/released?

A. A very good final question, but one that we must give a very broad answer to. We're still finalising a number of systems for the upcoming version but we're getting closer to the exciting day when we can reveal the full details of the next generation of Trainz. Thanks for your patience while we ensure this next release is our best ever.

Glass half full = sounds like it's getting quite close.
Glass half empty = sounds like we have to wait longer to find out.

We hope this "glass-half-full" vs "glass-half-empty" look at Trainz has given you a deeper insight to things behind the scenes here at N3V Games and the different ways of viewing our progress.