In the spirit of openness and sharing more information, we want to take some time and discuss MCC flighting and why the Halo: Reach PC “FireFlight”, which was originally planned for July, did not happen yet.
The short answer is that we chose to hold back FireFlight due to severe blocking bugs. While we continue working to address and solve those bugs, the challenges proved insurmountable to release the flight in July. Our goal when preparing for flights is to ensure that the experience hits the necessary quality bar to deliver an effective and successful flight for both Halo Insiders as well as 343 Industries. While we strive to ensure we run our flights on a timely basis, ensuring the effectiveness of the flight must take priority.
However, we want to provide everyone with the long answer, too. Today, we’ll share some insights into how flighting is handled, where development stands with respect to flighting, what work we have left to do, and why (while unfortunate) the decision to delay a flight is made in the best interest of our players and the game. Before we dive into development details, let’s look at what the process of bringing a flight to Halo Insiders looks like.
WHAT’S A “BLOCKING BUG”?
“Blocking Bugs” are “blockers” in that they are deemed severe enough to prevent a flight from functioning as needed. Keep in mind that though we address, mitigate and solve “blocking bugs” for each ring of flighting, this doesn’t mean that any flight is bug-free. In fact, it’s quite the contrary since each flight is very much a work-in-progress development build. In most cases we will be aware of existing minor bugs and proactively identify these known issues for Halo Insider participants, while in some cases it’s the Insiders themselves who bring new bugs to our attention while playing.
Each flighting build must successfully complete a series of tests before it’s made available to Insiders. These tests require that a build progresses through what we refer to as “Rings.” There are four rings in total with the final, “Ring 3” being Halo Insiders. For a flight to go out to the Insiders, it must make its way through each of the rings without bugs that are deemed blocking issues. Let’s go in and see what each ring is made of and the process that is followed to move from one to the next.
Ring 0 is made up of internal studio folks. This is a combination of the 343 Publishing Team, 343 Pro Team and developers within the studio who participate in playtests we hold regularly on the content we are aiming to flight. If serious bugs are found in Ring 0, it does not progress to the next ring and will not make it outside of the studio. The team goes back to the drawing board to address any issues that are identified. If no serious issues arise, however, we move to Ring 1.
Ring 1 is made up of external partners that work closely with the studio (many of which have for quite some time). This allows us to work with players who know Halo content and are spread around the globe testing our various pipelines and supporting services to ensure the experience is fully functional. If any major issues are found here, it can cause a reset all the way back to ring 0 if severe enough and warrants the build to be properly vetted and tested again. If all is well, this moves on to our next group of external partners.
Ring 2 is made up of even more partners of various skill sets that help test our builds in unique and interesting ways to ensure that we are providing the best experience for all varieties of users who are invited to test flighted content. If nothing show stopping is found here, it then will make the jump to our final Insider’s ring, Ring 3. As with all previous rings, if any major issues are found here, it can cause a reset all the way back to ring 0 if severe enough.
Ring 3 is the Halo Insider user group that is determined based on KPIs (Key Performance Indicators) the team sets for wanting to test in any given flight. This audience could be hardware based, experience based, locale, or any specific metric the team is looking to validate or stress at any given point as these flights are not BETAs, but indeed are at-scale tests to validate the state of the build to assess functionality and release readiness.
WHAT RING ARE WE AT RIGHT NOW?
At the time of writing this, we have just completed our first Ring 1 test for Halo: Reach FireFlight on PC. Late last week, the team resolved the final key issue that had been blocking entering Ring 1 and now has a few bugs being worked on that are blocking Ring 2. There are currently five Ring 2 blockers, and fifteen Ring 3 blockers the team is working through.
The five current Ring 2 blockers are as follows:
- Security solution does not auto-install once game is installed
- Long delays in menus accompanied by matchmaking errors can happen
- Opening the settings menu during gameplay can cause the roster to stay stuck on screen
- Frame loss can be experienced in various menus when played in higher resolutions.
- Crash is present sometimes when a matchmade Firefight game starts
The team is currently in progress working through all of these blocking issues with the goal of having them fixed as soon as possible in order to progress to Ring 2 flighting. Before we take the flight to the Halo Insiders though, we have a handful of bugs to resolve that involve navigation issues, crashes, and other key issues we believe will hurt the experience of the flight (both for players and for our team’s ability to gather critical data and feedback).
These key issues are blockers in that they are deemed severe enough to prevent a flight from functioning as needed. In most cases we will be aware of existing minor bugs and proactively identify these known issues for Halo Insider participants, while in some cases it’s the Insiders themselves who bring new bugs to our attention while playing.
NEXT STEPS PRIOR TO THE NEXT HALO INSIDER’S FLIGHT
Even with the Halo Insider’s FireFlight flight being in development, we have many other things the team is working on. Rather than derail core development and putting everyone onto the flight for bug fixing, the teams working on MCC PC and Halo: Reach for MCC are still focused on the deliverables they are working on. But it’s safe to say that these core pieces are needed in order to properly deliver the best PC experience when its ready to release.
As each blocking bug is fixed and new ones are found, we will continue to move through the rings appropriately and get ready for our next Insider flight. We have not and will continue to not publicly announce dates of when this will happen as we are actively in development of the flight. As we have stated before, the team is heads down on working to resolve these issues and continue development on MCC to PC and Halo: Reach to MCC. As we’ve seen thus far, often times even our best estimates end up shifting as the team works through the production and development realities of this complex project.
When the Firefight matchmaking experience and all content is ready to flight, it will flight. It’s the same mantra we have had from the beginning and will continue to have. As with any product that is actively being developed and having a high level of transparency, things change. Dates will move as needed, estimates will be accurate and inaccurate, and goals will be hit or missed. Nobody can predict the exact future, and this is what comes with the territory of being as transparent as possible with the development process as it relates to a AAA game title. We will continue to do our best to keep you in the loop and up to speed – we know everyone is very eager and anxious to jump in.
WHY FLIGHTING LATER MAKES SENSE
There seems to be a lot of public perception that as flighting goal dates have been missed that the project itself is going to suffer in the quality department, but that’s the opposite of what the reality is. In order to meet the high-quality bar the teams have set for this project, these delays are to ensure it is the best it can be. A lot of development work in bringing Halo to PC has been invested in building the core experience of the user interface, controls, and how the game plays on PC so it feels right at home on the platform. To put it simply, we’re placing an extreme focus on the foundations of MCC to ensure it’s authentic to the Halo experience andthe PC platform.
If quality wasn’t something that was cared for, a mediocre product could be shipped quite easily. But we don’t want that and we know you don’t want that. Every stakeholder for this game, including you, the teams involved; all want the best possible product and therefore are making the right decisions to hold back a subpar experience. If something is held to a later date for a better experience it is worth it tenfold. We feel strongly that this approach is worth it, even when it lands us in hot water with some of our fans. Here are a few examples of some bugs that have been addressed over the past several weeks on the FireFlight builds.
- Upon launch, the user was prompted with a DLC prompt to install Reach and was unable to play
- Navigating too fast to the Matchmaking Menu would display an empty menu
- Settings would be discarded when changing them during gameplay
- A crash occurred when entering the setting menu
- Disabled roster buttons were accessible and caused crashes sometimes
- Title crashes when pausing in Reach gameplay using a controller
- Game has a fatal error when saving settings during Firefight match queue
All of these have been fixed with many others as well, but just these along being fixed has improved the quality of the experience significantly. The team is dedicated to providing the best experience possible to users and is working diligently on the experience for not only the next flights, but eventual release.
WHY SO COMPLICATED?
When we announced MCC coming to PC and Reach coming to MCC, we thought we knew exactly what the roadmap was to bring about the best game for gamers. We were wrong about a few things and this is something that happens often in the complex, fluid world of game development. To compensate for this, schedules around flighting had to be shifted out. But, even with these challenges, the development team has overcome them and is continuing to work on it day-in and day-out towards bringing MCC to PC and Halo: Reach to MCC the right way.
Announcing this project back in March was also tied to giving ourselves time to establish the Halo Insider program. We overestimated the time it would take to build up a sizable Halo Insider audience to tap into for our flighting programs. We erred on the side of announcing earlier to ensure we could get the program up and running. The Halo community showed up in overwhelming numbers and we shattered our high-end opt-in goals in the first week. Thank you for the incredible level of interest and support!
In the games industry, schedules are an attempt at a best guess of what is needed to make a video game. With 13 years in the industry, I have never seen a schedule mirror exactly what was originally outlined in terms of scope or time spent on building it. The only way to make a schedule work within a specific constraint requires one of two things – One is to cut content and either move it out or remove it entirely. The other is to sacrifice quality.
We don’t want to do either and therefore flighting has moved out slightly to not impact the core product we are aiming to build. This is the best option for all involved. Does it mean people have to wait a little bit longer? Yes. Does it mean a worse product is coming? No. It means the games are being developed with the highest quality bar possible in mind and to ring that bell again, it’s ready when it’s ready.
Now that we have all that fun stuff out of the way, what are we looking to do for flighting all up? The same things as previously stated before in the July development blog HERE. We plan to flight Firefight Matchmaking and a PvP Matchmaking once these flights are ready to go for PC.
And what about Xbox? The goal is as soon as the Xbox One version of Halo: Reach is ready to go we will flight that. For PC, FireFlight is on the horizon and is our next Insider flight. When will it happen? When it’s ready. Some players have been upset by that phrase, but it’s the best attitude you could ask for from a developer as it truly shows how much quality is cared about over dates.
We hope this lifts the veil back even further and gives a much more in-depth look at what’s going on specifically with Halo: Reach flighting. Thanks again to the 343 Pub Team for the go-ahead to break this all down and share even more in-depth specifics of how things are tracking and what’s going on! Until next time, we will see y’all online and look forward to feedback and questions for future blogs and communiques such as this.
If you’d like to join us on this journey, be sure to sign-up with the Halo Insider program HERE!
For additional insights and details on the MCC PC project, please check out our previous monthly development update blogs listed below:
- MCC PC & Halo: Reach Announcement
- MCC Development Update – April 2019
- MCC Development Update – May 2019
- MCC Development Update – June 2019
- MCC Development Update – July 2019
We hope that this helps share more information with everyone and gives more insight into development. Also as a bonus here is a pretty image of a work in progress render for the Elite Minor in Halo: Reach’s armor customization screen. This was shared to me from Wrensi over on the Publishing Team. We will see y’all online!