Hello yet again! Yep, I’m aware that it has been all interviews recently (I should be working on proper articles soon); but today I was able to interview someone pretty interesting: You see, if you have been in the emulation scene recently, then you should have heard about the rise of a promising, brand new Xbox emulator in these years.
The Xbox console, made and released by Microsoft in November 15, 2001, was a pretty interesting contender in the sixth generation of consoles, as it counted with several games from studios often associated with PC games, as well as being the debut platform for the Halo series, which both the first and second games acted as killer apps for the Xbox, on top of counting with several exclusives (most notoriously, direct sequels in certain SEGA franchises spawned from the Dreamcast), as well as introducing the Xbox Live online platform, allowing for networked multiplayer over the world. The Xbox also allowed connecting Xbox consoles locally through System Link, similar to an LAN setup with computers, which was used to great effect by enthusiast for shooters (the most obvious case being Halo 2).
With the Xbox specifications being actually mostly based on computer components, with a customized 733Mhz Pentium III processor, a customized NVIDIA GPU, and 64MB of RAM, it was assembled with quite powerful hardware for the time (both for consoles and, in comparison, computers), and coupled with the use of the evolving DirectX technologies (which Microsoft had made for Windows, but made a modified version for the Xbox API), it showcased a lot of potential. Of course, with the similarities to a PC, it would also receive a huge modding community since its release.
However, while it was mostly based on computer hardware, it wasn’t PURELY computer hardware, and the software was mostly different from standard Windows, so it really was not an stock computer in console form; but it actually was almost there. Around 2003 (2 years after release), Caustik would develop Cxbx, which worked around the concept of patching Xbox function calls to work on the Windows equivalents, similar to a wrapper instead of an emulator; but it would be quite difficult to progress (and didn’t get working audio on that time), eventually getting halted. Since then, there was a notable absence of Xbox emulation efforts, not only because of the Dreamcast, Gamecube and PS2 consoles being emulated through NullDC, Dolphin and PCSX2; but also with the years it would take for this new project to surface. While a Xbox emulator did show up, XQEMU, at that point it was much more focused in accuracy and documentation; so wihle it was an admirable effort and something that would help later on, it was quite heavy among the need of optimizing, and the full emulation approach. But there would be another option in progress: Cxbx-Reloaded started in 2016, and since then, it would take that initial idea from Cxbx, but develop on it with newer knowledge, optimization, and of course, more work on functionality; reaching quite playable states for some games between 2017 and 2018.
Cxbx-Reloaded would be a welcome surprise in the emulation community, as not only it was another breath of fresh air in this previously unconcluded story, but it also brought hype to many Xbox fans (and people that never were able to have one) as games were showcased to be in high-performance playable states, with audio and controller support, and unofficial upscaling methods. It indeed was a light of hope for those Xbox enthusiast, and a glimpse of what could the future would bring for the Xbox emulation scene. With that said, this is the reason why I actually went out with a long introduction for this interview, as the milestones that Cxbx-Reloaded has been reaching has been the culmination of excruciating efforts, to finally be able to run Xbox games after how the other consoles, and even later consoles like the PS3, Xbox 360 and Wii U, were already in steady development, if not matured enough to play most games.
The main man behind Cxbx-Reloaded, Luke Usher (known as SoullessSentinel all the way back in the NGEmu forums), provided some interesting stories about how Cxbx-Reloaded actually came to be, as well as the conditions that brought it to fame, and a explanation about Cxbx-Reloaded approach for playing Xbox games (he also currently works on Insignia, an OG Xbox Live replacement; though at the time of this interview it was only hinted). Without any further ado, please enjoy!
This interview was conducted on the date of April 30, 2020 and is mostly presented as it occured.
Since how long have you been in the emulation scene?
Luke Usher: I’ve been using emulators for a very long time, since I got my first PC way back around 2002, but as a developer, Cxbx-Reloaded is my first emulation project, and that was started around 4 years ago.
What inspired you in working in Cxbx-Reloaded?
Luke Usher: Oh, this might be a long one.
Way back in 2013 there was one particular night where I couldn’t sleep, for no particular reason. It was late at night, and I was looking for something to occupy my time. I remembered there was an old Xbox emulator (Cxbx) that I hadn’t heard anything about for quite some time. I decided to check it out, but very quickly found out it didn’t work on modern computers.
That started an investigation into why, and I began working on fixing the problem. You can see an old thread where I was talking about this over on a ngemu thread.
I didn’t know what I was doing back then, and naturally I didn’t get very far. I managed to get Cxbx working, but it was very unreliable and unstable, ultimately, I abandoned it and deleted my code.
Then about 4 years ago, I’m not entirely sure why, I decided to give it a second attempt. This time I had more success and my experiments evolved into the Cxbx-Reloaded we have today.So, to answer the question, it was really accidental/luck, there wasn’t actually any inspiration to start with. Interestingly, I was never even an Xbox fan!
Do you have a favorite out of all the Xbox games?
Luke Usher: Like I said, I wasn’t really an Xbox fan. Truthfully I still haven’t played any of the games properly, except for the testing I’ve done while working on Cxbx-Reloaded. Probably not the fun answer you were hoping for, but hey, at least we’re not biased towards supporting particular games 😛
I’m aware that Cxbx-R originated from a previous project, Cxbx. What can you comment about that initial version?
Luke Usher: Cxbx was interesting because unlike traditional emulators, the goal wasn’t to emulate the Xbox hardware, instead, it worked by converting Xbox executables to .exe files, and intercepting functions that would access the hardware. In a way, it’s something rather similar to how Wine would run Windows software on Linux.
Amazingly, it was able to get Turok Evolution playable all the way back in 2004! But sadly, development on Cxbx got halted, and some few people attempted to pick it back up, but ultimately, new releases never surfaced.
Can you comment about the emulation scene around those years? (Since or before the original Cxbx project?)
Luke Usher: I wasn’t really around at the time, back in 2003 I would’ve been around 10/11 years old, all I knew about Emulators was you could use them to play games; so can’t really comment on the scene at the time.
Around what time do you think or remember that Cxbx-R started taking off in popularity?
Luke Usher: I think it got noticed pretty quickly after it started: There was very little Xbox emulation in the news, XQEMU was around but it hadn’t gathered a lot of awareness yet, people got excited as soon as they saw Xbox games booting on modern computers.
I gotta say though, we didn’t really get ‘big’ until Jet Set Radio Future started to work, people really love that game.
What were the first milestones planned with Cxbx-R?
Luke Usher: There wasn’t really a solid plan, we just kept fixing issues as we found them. The first, and most important was to solve the problem that prevented Cxbx from working on modern computers.
Without getting too technical, original Cxbx relied on the fact that 32-bit software could alter the memory layout of a table known as the LDT. On 64-bit windows, it is no longer possible for 32-bit software to control this, which meant Cxbx just couldn’t boot games.
After that? It’s been a near constant loop of running games, spotting a problem, debugging it, fixing it, and trying another game. Occasionally we run into complex problems that the design of Cxbx/Cxbx-Reloaded wasn’t really designed to handle, but we always find a solution eventually.
I must stress though that we don’t work on specific games, rather, we try to work out what the correct behaviour would be on a real Xbox, and if we replicate that well, games eventually start to work.
You can see this in action pretty regularly, as we get reports of game that we have never tested working.
What was the first game you can remember that was functional/playable in Cxbx-R?
Luke Usher: Because we built Cxbx-Reloaded off the Cxbx project as a base, Turok Evolution was one of the first games that worked from the start. I think Futurama was one of the first games I was able to play well.
But Jet Set Radio Future was the first game anybody seemed to care about though, as despite a lot of successes, we never really took off until we could show that game being played.
Can you provide an explanation about how does Cxbx-R emulate the Xbox? (A simplified approach and/or an technical resume of how things work? Would be good to cover both something for those initiated in technology as well as with more experienced people in emulation and hardware)
Luke Usher: Traditional emulators work by simulating the hardware of the system they’re emulating: they usually emulate the processor, the graphics hardware, the sound hardware, and everything else, as far as the games are concerned, they are running on the original console.
Cxbx-Reloaded is completely different, in fact, it might be incorrect to call Cxbx-Reloaded an emulator, it’s probably more correct to refer to it as a compatibility layer. Cxbx-Reloaded does not emulate an Xbox, nor it doesn’t emulate the Xbox hardware. Instead, it loads Xbox games into a Windows process, locates all the Xbox specific functions, and patches the games to use our own versions of those functions, (mostly) avoiding the need to actually emulate the Xbox hardware.
If there are any folk around that are familiar with Wine, an application that allows Windows software to work on Linux, Cxbx-Reloaded is almost the same kind of thing, but for running Xbox games on Windows. Cxbx-Reloaded lets you play Xbox games on your PC, but not by emulating the Xbox hardware, instead, it emulates the software environment.
This does have a downside in terms of accuracy, as it will probably never be identical to running games on an Xbox, but it has a lot of upsides too: We gain a lot of performance by not having to emulate hardware, being able to run natively makes it easier to modify games too, you can even use tools designed for hacking PC games such as “Cheat Engine” with Xbox games without an issue. We can even run games at a higher frame-rate than the Xbox would support: the games can use the full power of your hardware, instead of being restricted to the limitations of the Xbox console.
Hmm, very interesting. You think there is still room for improving the performance? (or better phrased, what do you think that will be needed for improving performance?)
Luke Usher: Performance is excellent in many games, on even low-end hardware, we’ve seen games running at much greater than full-speed. I’ve seen over 1000 fps in simple scenes myself. Sadly, there are also games that struggle even on the best computers. Cxbx-Reloaded is still pretty slow in a lot of areas, many of which are to do with fundamental differences between Xbox and PC architecture.
Xbox games tend to use vertex buffer formats and texture formats that are not natively supported by modern graphics hardware or APIs. Converting these resources is a slow process, but that’s okay, we can cache the converted resources so we don’t need to convert them every single time they are used, but only if the contents change.
Unfortunately, there is no easy way to detect changes. There is no API we can hook to detect when software will change a resource: Unlike PC, Xbox doesn’t have functions for Locking/Unlocking resources for access, so games are free to alter data at any time during execution. Since nothing will tell us that data has changed, we need to do all the checking ourselves.
Currently, this is done by hashing the contents of textures, vertex buffers, and similar resources every time they are used, but this is very demanding on the cpu. We really need to find a better way to handle this.
Windows does provide a feature that can do ‘page tracking’ in which the operating system can notify us when the contents of a memory region change, this would be perfect for us, except it doesn’t work with Cxbx-Reloaded due to quirks of the Xbox memory system that are incompatible with this windows feature.
Converting and hashing data is one of the slowest parts of Cxbx-Reloaded, so a lot of effort has gone into making it as efficient as possible, but there is still room for improvement.
Would you say that the hardware you currently have for developing Cxbx-R is good enough? Or would it be better to have something improved than your actual setup?
Luke Usher: I think it’s been very beneficial to be doing my work on Cxbx-Reloaded using my outdated laptop: It forces us to optimize as much as possible to try to get as much performance as we can, which is one of the reasons I’ve been quite reluctant to upgrade during development. However, I was forced to upgrade only a couple of weeks ago when my laptop suffered a cooling failure, so now I have a shiny new laptop.
Well, it is nice to know that both it helped on optimizing as much as it was possible, and that you got a new laptop 🙂
What would you say that is the reason that it took so long to get to this state in Xbox emulation? (or in the case of Cxbx-R, running Xbox games on PC?)
Luke Usher: A lot of people think the Xbox is just a PC, or that it has no exclusive games worth playing, so there hasn’t ever been that much focus on emulation until quite recently, and since the Xbox is pretty old at this point, it still doesn’t get much interest.
Looking at Cxbx-Reloaded in particular, we have only existed as a project for around 4-5 years, and we have managed to get almost 80 games fully playable in that time, despite only having a small handful of developers.
What can you say about the rest of the Cxbx-Reloaded team?
Luke Usher: They’re great, not much to say other than that really, they’re just as dedicated to the project as I am, lots of great work going on. PatrickvL is an experienced software developer with lots of experience optimising code, so he always has good insight on where we can squeeze out some extra performance. RadWolfie has done an incredible job with the sound emulation, in fact, almost all the sound support was implemented by him. Ergo worked hard on our current input and xbox memory map emulation. NZJenkins is one of our newest contributors and is doing incredible work in the graphics department.
I’m sorry if I forgot anyone here: we’re an open source project and anybody can contribute, so the ‘team’ is really anyone/everyone.
blueshogun96 was the original Cxbx dev, right?
Luke Usher: That’s a common misconception: The original developer of Cxbx was Caustik (aka Aaron), blueshogun96 did contribute though.
None of us have ever heard from Caustik, he seems to have completely left the scene, but we do hear from blueshogun from time-to-time, he’s often very supportive and complimentary of our work.
Did you have an Xbox for reverse-engineering or debugging to work on Cxbx-R?
Luke Usher: Not at first, most of the work is done on the games themselves: thanks to the way Cxbx-Reloaded runs games, we can step through the assembly code of Xbox games using Visual Studio, just like Windows software!
Outside of anything Xbox-related, what was the emulator that inspired or impressed you the most?
Luke Usher: There’s quite a few. MAME will always be impressive just for the sheer amount of research and systems they support. Going back in time now, we had Virtual Game Station, which was a commercial emulator that could play PS1 games on hardware as old as the iMac G3. In modern emulation, the project that impresses me the most is Orbital: An attempt to use virtualization to emulate the PS4 hardware, and they have succeeded in booting the firmware into safe mode.
What are your thoughts about the other Xbox emulators (XQEMU and other ones, if there are any)?
Luke Usher: XQEMU is a great project, but it has different goals than Cxbx-Reloaded.
Where Cxbx-Reloaded is designed to play Xbox games on your PC, XQEMU is designed to be a virtual Xbox, and while it’s not really suitable for playing games at the moment (but it will be eventually), it is incredible as a research project right now.
The XQEMU developers are one of the only set of people who are actively researching and documenting how the Xbox works from a hardware expensive, and it’s a very important work. I fully support them, and we don’t see ourselves as competition due to the different goals.
What are your feelings on the current state of Cxbx-R (and the whole experience behind developing it all these years)?
It’s been fun and rewarding to see more and more games working over time, and this year is looking to be a very positive year, we’re getting close to having 10% of the game library playable.
Luke Usher: Cxbx-Reloaded has developed to the point that it’s even helped with the game modding scene: neodos uses Cxbx-Reloaded to help develop/test mods for Jet Set Radio Future, and it’s amazing to see our work used in such ways.
I have seen many people work with JSRF mods recently, it is very nice indeed. What have you learned with all the work you’ve done so far in Cxbx-Reloaded?
Luke Usher: Almost everything I know about programming for Windows came from working on Cxbx-Reloaded.
When I started I didn’t know a single thing about how the Windows APis worked, I didn’t know anything about x86 assembly, the xbox hardware, or any of that stuff at all, it’s been a very big learning project for me, and that’s what makes it fun.
Do you have any other personal projects?
Luke Usher: Quite a few, but most of them I’m not ready to talk about publicly just yet, one of them is Xbox related and will hopefully be very exciting for everyone, so expect more news on that soon.
April 2021 editor update: As of May 20, 2020, this hinted project was revealed to be Insignia, a currently work-in-rogress replacement for Xbox Live servers that works on Original Xbox hardware (and probably will/should work with Cxbx-Reloaded as well :p)
That sounds great! On the note of side projects, I remember that you were working at one point on a PS1 emulator, PolyBlast; but was cancelled later on. Any comments about your experiences with that?
Luke Usher: I wouldn’t completely say it’s cancelled, but more sleeping. Other projects came along and did most of what I wanted to do with Polyblast, but did it better and faster than me, but in the end I learned a lot from it, so I still consider it a success. There’s a chance it may be revived in the future, but probably not anytime soon, since that right now, there are other projects that have higher priority for me.
April 2021 editor update: As of February 23, 2021, Luke officially released PolyBlast v1, now as a multi-core emulator like ares/higan and collaborated with Near for several of the cores, including Playstation 1 support.
Understandable. Something that I forgot to address a while ago, was the work-in-progress SEGA Chihiro support; what can you comment about that? And not just in the progress made, but also, what would you say that is different compared with stock Xbox hardware, and what extra features need to be implemented to make it work?
Luke Usher: For those not aware, the Chihiro is an arcade system developed by SEGA, based around the Xbox hardware. It consists of an Xbox debug motherboard (which means it has 128MB of ram, where a normal Xbox only has 64MB) and some SEGA custom arcade hardware for interfacing with JVS cabinets and loading games securely from GD-ROM. (yes, the Chihiro uses GD-ROM like the Dreamcast and Naomi, not DVD)
An experimental branch of Cxbx-Reloaded can partially emulate the Chihiro hardware: enough to boot a few games, but not quite good enough to play them yet, but thankfully, most of the issues are issues that also impact Xbox emulation, so it will improve over time as the emulators gets developed.
The issue preventing the Chihiro work from being made available in the main branches is simply that we don’t support game-specific input devices yet. Unlike the Xbox, each Chihiro game uses an entirely different input configuration, so we can’t just emulate a single controller and call it ‘done’.
I assume that this is related to arcade input standards, like JVS; but also related to input methods like steering wheels and the sort (or a skateboard, for Ollie King) working with non-standard devices (as I’m aware that Cxbx-R, outside of keyboard, supports Xinput out of the box)
Luke Usher: Yeah that’s exactly it. The chihiro emulation does emulate the JVS hardware, but not all the game-specific button mappings are implemented yet, and not game-specific hardware like the drive board for Outrun.
What can you comment about the System Link implementation? I saw that it has worked for Tony Hawk Pro Skater 2X so far.
Luke Usher: It will work for every single game that works in Cxbx-Reloaded, there’s not much to say other than that: Cxbx-Reloaded emulates the Xbox network hardware, so as long as you connect an ethernet cable to your computer, it works exactly as a real Xbox would on your network.
There’s a slight limitation in that, as in order to cross-play with real Xboxes, you need a copy of the Xbox encryption keys, but once you have those, it just works, and we didn’t have to do anything special to support it.
What could you say that would be the toughest games to see working in Cxbx-R so far? (based on hardware quirks or unimplemented features; excepting anything related to Chihiro)
Luke Usher: Halo has been surprisingly hard to get to work, and we’re not entirely sure why: For all purposes it just seems like a simple launch title, but no matter how much effort we’ve put into trying to get it to work, we haven’t had success just yet, it’s probably just something very simple that we’re missing, but it’s driving us all mad!
What did you think when I reached out to you for this interview?
Luke Usher: I thought it was interesting, but I was a little bit worried that it wouldn’t be interesting for the readers, or that I’d end up saying something stupid and making a fool of myself. I tend to do that a lot 😛
Heh, it is always fun to be able to talk with people as talented as you, as well as someone that has been working in something like Xbox games on PC after the amount of time that had been completely unattended. Really great job!
Do you want to give greets or special thanks to anyone?
Luke Usher: I think I need to thank my partner for putting up with me working on this project, she’s been very supportive; and of course I’d like to thank our users too.
Do you have any message to the emulation community (as well as the Cxbx-R users)?
Luke Usher: For the emulation community and our users, I’d like to thank them for their support. Cxbx-Reloaded would not be successful without you!
And what do you think that will be upcoming for the future of Cxbx-R? Something missing or planned?
Luke Usher: For Cxbx-Reloaded’s future we’re still focusing on game compatibility: We’re trying to get as many games working as possible, and I think we might even pass the 100 working games mark pretty soon.
Got anything else to add? (personal experiences, a last message, whatever you want)
Luke Usher: I’m very bad at such open-ended questions, I don’t really know what to say.
I guess I’d like to thank you for taking your time for this interview, and I hope at least somebody enjoys the content 😛
Thank you so much for all of this!
Luke Usher: No problem! *thumbs up*
With all this said, Cxbx-Reloaded is really promising, not only in the approach of performance for Xbox games (thanks to the way it works, acting as a wrapper instead of a full emulator) that would allows lower-spec hardware to use it, but also on that these are finally playable outside of the Xbox consoles after such a long wait. Along with Xbox homebrew and the Xbox dashboard being very functional, exclusive games like Panzer Dragoon Orta, SEGA GT Online, Dead or Alive 3, Virtua Cop 3 (which is actually a Chihiro game; except it was designed to work on a debug Xbox instead of an actual Chihiro, meaning it can be played with the standard controller), and the often requested Jet Set Radio Future are currently playable, and many others are either ingame (but with some details) or halfway there, but the progress done considering both the time and the approach are admirable; and with development being constantly in-progress, it will definitely be worth keeping a eye on Cxbx-Reloaded and what it can bring in the future.
Thank you for checking this out; if you enjoyed this, you may want to check out the other interviews, and stay tuned for some interesting articles upcoming; specially if you are a emulator enthusiast. If you want to see something specific covered here, please let me know on the comments, and support me by sharing this to your fellow gaming friends!
And if you would enjoy my stuff here getting adapted to videos, you can share what would you want to see covered first and support me on Ko-fi! And if you feel you have already seen this one before; you might have, currently updating several interviews to the current simpler format so that there’s consistency between the older ones and recent ones, as preparation for future content I will be looking forward to bring here 🙂