The BeOSJournal spoke privately with Jonathan Thompson regarding his upcoming Project 'X', which aims to be a RAD/IDE development tool for BeOS.

Jonathan recently asked the community for feedback, with a poll, regarding where to focus development, feature-wise, and now this interview should bring forward some of Jonathans' thoughts on how the project will proceed.

Welcome to our interview, Jonathan!

Thanks, Chris! I'm more than happy to provide answers to many questions to answer the concerns of the BeOS community. However, I cannot and will not answer all questions as of yet. Wait for the release!

Can you describe what you mean by 'unusual' design interfaces?

The unusual GUI design interfaces involve using the best of both worlds of the GUI portion with a command line interface: both are integrated in function, and everything is possible (short of needing to type text to create something) without requiring a mouse to get there; the GUI is designed for the fastest keyboard navigation possible, with great care paid to tab order. The underlying design is meant to be maximally functional with minimal user distraction, with all visual detail created for showing useful information, with minimal functionless eye candy.

The GUI design of "Project X" will (as far as I can do it) allow anyone with any level of vision to use it in a productive manner; as many as 1 in 12 people have color perception abnormalities (I don't personally, but I've worked with many that do), and many more people than that have other visual perception problems. Thus, I'm designing the GUI to be functional in black and white, as much as possible, such that color is just an added way to communicate the same information.

Usability and efficiency as well as stability are the design goals. The user interface will be as orthogonal as I can figure out how to make it, which will also result in less code writing and debugging for me.

Do IDE environments today not work as well as you want?

Take Visual Studio, for example: it's a very powerful IDE, but the power is often inefficient to use due to the complexity of the interface. You need to remember either a very large number of keyboard shortcuts, or you are stuck using a mouse. When it comes to creating new variables, derived classes, methods, etc. in the class wizard, you have to navigate a rather complex dialog and you are stuck working within that dialog until you're done, with no way of looking at something you might be interested in for a reference.

If nothing else, the dialog box obscures your vision of what you're working on, and having to enter a complicated dialog box as done by typical IDEs involves a major distraction, even if things work right.

Also, with all the IDEs I've seen, all generated code is inserted in an order forced by the IDE, with no way of changing it, short of editing it by hand; this defeats the point of using the code generator in the first place, since you'd hope it would make things more efficient.

Can you elaborate more on your computer background?

I'm one of those people that reads OS source code for entertainment purposes ? I have over 20 years of experience, starting with TRS-80's with cassette tapes, and learned 6502 machine/assembly language on the Apple 2 series, and I earned my high school varsity letter via the computer club programming competitions using Apple ][e's with AppleSoft BASIC in a format that resembles extreme programming as seen today. I assisted a friend down the street that wrote Apple 2 software on writing a ProDOS disk defragmenter in the late 80's, when people were just starting to have hard drives for their computers.

I wish Apple had continued with the development of the Apple 2 series, as there was plenty of room for the processor family to grow, and the Apple 2GS was way ahead of its time in both the power of the OS for the hardware it ran on, along with the hardware itself. The 65C816 was the closest thing you'll find to a RISC processor with 23 addressing modes, though some of them were purely optimizations for dealing with 8 bit limits. However, it could work with more than 64K data with instructions without going into segmentation like the Intel processors, and could address 16 megs without a problem. Do I sound wistful about those times?

Other than that, my professional experience has been writing software for mission critical applications, in terms of the fact that time and reliability of the process were not merely important, but vital. This includes low-level experience writing for SCSI devices such as CD-ROM drives, tape drive and CD-writers for CD-ROM pre-mastering utilities working for Digital Audio Disc Corporation, as well as being part of the team that created the first mastering machine solution to creating production multiple session discs (Surely people have at least one of those in their collections, the ones where an audio player only sees the first track(s) as audio, and doesn't see other sessions, thus avoiding listening to application code).

I also wrote an application to analyze CD's at a very low level for adherence to standards that first started out in Windows with heavy C++ object oriented design, and then (at the demand of the project leader) found myself having to port it to ANSI C under Linux using a multiple process design instead; thus, I have some rather interesting experience with knowing what a C++ compiler does for you and to you! The next employer had me writing software for an engine monitoring system, collecting and analyzing telemetry.

My most recent employment involved writing software for controlling CNC milling machines and press brakes that aren't shut down unless they break or need repairs, and the software was designed to be that boringly reliable. I've been writing software in C/C++ since 1992, and have collected a lot of arrows in my back, used several different OS's and development environments and APIs, and worked both in teams as well as individually.

Are you interested in help from other developers if anyone stepped forward? Have you had any offers already?

I've had offers for beta testers to help me already, but I'm not ready for that yet anyway. I've had more questions of why I haven't helped on other existing projects. Other than having beta testers verify "Project X" works as desired (I'm confident it will be very stable, but everyone has their own idea how things should work, and anything this complex will have at least minor bugs before release, as well as likely after) I have no real desire to have others help me with the code itself, though there might be questions I might ask about doing a few things here and there.

I have a clear vision of what I want to do and why I want to do it, based on personal experience and a certain amount of market research over time on BeShare, where I've previously logged in as coolbear; coolbear is an ancestral name I've used because it's a lot more likely to have fewer name collisions on the web than my legal name. For those that have suggested features in the distant past on BeShare, be assured, I have not forgotten them!

How fleshed out is your design now?

I've got about 30 typed pages of description of the structure of how it should work and look, as well as the threading model that goes along with it. I've had this on my mind for over 3 years now, and I've waited more than patiently for other projects to get complete enough to fill my needs, but I could wait forever if I don't do it myself. The BeOS GUI API with the requirement of having a thread for every window and the application itself makes it a non-trivial task for ensuring that things don't get created and destroyed in such a way that avoids deadlocks and crashes due to invalid pointers.

However, I have a solution that guarantees windows are not created or destroyed unexpectedly, and the manner of controlling them ensures no deadlocks as a result. Hopefully, this won't be regarded as rocket science to everyone, but just good common sense. I anticipate releasing a minimal weight document/view library with "Project X" that demonstrates by real code how that works, and I think it will help quite a bit.

How much experience do you have in programming for NT, versus BeOS? What are the most appreciable differences?

I started NT programming in 95 while working on a CD analysis system for Sony, and had to move over to Linux, due to some low-level I/O subsystem flaws in NT 3.51 that quickly brought up the BSOD when I asked it to do something it said, "Yeah, I'll do that!" when in actuality, it couldn't. This forced a switch to using Linux, because we needed to modify the SCSI subsystem to allocate and page lock HUGE buffers, due to some interesting CD-ROM drive and CD limitations. NT does not allow a large amount of memory to be page locked at any one time; BeOS is much more flexible in that regard.

NT is very much a heavy API system in terms of going through the formal channels of trying to make everything fair to the other applications and the system for scheduling, and has a horrendous amount of overhead as a result. If you want to scare yourself silly, look at the DDK for NT, and you'll quickly understand that it's amazing that as many drivers work as well as they do.

The API (GUI and non-GUI) is also a mix (at this time) of heavily C and Pascal based design and C type objects, mixed now with a combination of C++/COM objects that are melting in a huge pot of boiling oil and water, and they don't always mix very well. Creating a Windows GUI that's fully testable isn't something that's the easiest to do via scripting in a reliable way, or at least not with Visual Test 4.0, which is the version I had experience with (this was 2-3 years ago now, and companies are loathe to spend money on an upgrade if the older software works well enough) such that it can take awhile to create something that's inherently testable.

The Windows GUI API is a strange combination of having too much and yet too little information readily available to an outside program, or even inside the application. Because Windows allows you to resize all the GUI elements by the Control Panel, including all the colors, it can be a nightmare to create resources to present a GUI that works on all machines reasonably well. The resource editors and how Windows handles dialogs makes things somewhat manageable, but working in Dialog Units has its limitations.

Most Windows GUI code is either going to be messy straight C, or you'll likely use something like MFC or Borland's OWL or their newer variant that ships with Borland C++ Builder and Delphi, or you might choose a bunch of other frameworks variants that are cross platform. However, the dominant one now is clearly MFC, which is in many ways a thin wrapper over the C API, which gives some power while taking a bit more of it away.

For example, MFC didn't have templates available under the Microsoft compiler when created, and it shows. Also, you are pretty much stuck with the overhead of their Document/View architecture, unless you create an application that's only a dialog. It takes way too much effort to get a window up and going and useful in terms of the amount of code and complexity. Fortunately, Windows has no shortage of powerful and easy-to-use GUI Editors and development environments that are sufficiently stable.

When it comes to writing multithreaded applications for Windows, you're often better avoiding MFC, because it forces you to use Thread Local Storage in an awkward way that can create all sorts of problems. Windows itself wasn't initially designed to really work with multiple GUI threads within the same application; using a single window thread for all windows and having background threads do the nasty stuff works best.

And yet, the curious thing is that Windows provides more powerful threading primitives than BeOS does: it has process local, user space critical sections, which are much faster than the mutexes, semaphores and event objects, and it's easier to share them between processes (except critical sections, of course!) because of how they work.

There's also the newer thing called fibers, which seem to be the Windows way of doing light threads that are all within a single actual thread, sort of like a personal address space cooperative multitasking primitive that shares the same thread context. But what is perhaps the most powerful threading primitive under Windows is the ability to wait for multiple objects, and you have the choice of either acting on the first one that's signaled, or waiting for them all to be signaled.

When it comes to infrastructure of messaging between Windows and BeOS, Windows requires you to know about a lot of arbitrarily defined message IDs, and extract data of limited type and size from within a message. I like the power of BeOS for the richness of messages and how much you can expand them; however, Windows makes it easier to test applications with messages by their regularity for the common controls, and in Windows, there's not much chance of ever losing the simple messages even when the system gets very busy. It might run slow, and the GUI might not update as nicely as BeOS, but as long as it hasn't crashed, it will continue in a predictable manner.

One thing a lot of the Windows GUI API has that the BeOS GUI API doesn't have: extensive error reporting and reasonable failure modes. If you run out of GUI resources in Windows, you'll get back an error message when you try to create something. If you try to create another window in BeOS, including just the unstoppable action of displaying a menu that requires another thread, it's not pretty when the AppServer runs out of threads, or your application runs out of threads. There isn't an easy way to make an application resilient against such things beyond the process of the application itself in BeOS when it comes up against such limits.

While Windows isn't perfect, you can at least drop back to a fairly sane failure mode in the worst case. In BeOS, you end up with windows and applications that are hung or killed, and once you've reached the AppServer thread limit, each time you try to create another window (including menus) you've just hung another application. If you flood the system with too many messages and they can't be handled fast enough, messages start getting dropped by everyone, and there's no easy way to detect that it's happened, and no way to fully prevent it at least from affecting your BWindow and BApplication threads. If all your application state is managed via messages sent through BMessages and ports, your application is subject to the whims of the rest of the system far more than it would be under Windows, if only because you can't even detect via the API that your application is failing.

These BeOS system problems are some of the reasons why as much as I'd have liked to use it in my last project for my most recent employer for controlling CNC machines and press brakes; another reason is that the other needed things just don't exist.

I first personally discovered BeOS when version 4.0 came out, though I was introduced to it (at least in passing reference) before it was available for Intel, but I didn't get too involved with it at that time, due to lack of resources to do so. I've done mostly small things for BeOS itself, though I was working with Guillaume Maillard on Eden for a bit, helping him untangle some code and threading design structure problems.

Will 'project X' be compartmentalized enough to allow several small tools to be spun off separately as downloads on BeBits?

Yes, due to how I design and implement software, it defines "modular" by example.

Are you planning to go fully commercial after enough development, or will it be open source all the way? ;)

"Project X" is not going to be open sourced, and will be a fully commercial product with no restrictions on people reselling/redistributing their applications that link to any libraries I might distribute with it. I haven't completely decided what license to use for any external libraries I create, but it will NOT be GPL, as that's too restrictive for anyone that wants to sell their applications commercially.

How do you manage your source internally where you work, and at home? Do you use cvs tools, or anything similar?

At work in the past I've used SCCS, Visual SourceSafe (a system that can effectively scramble your source and keep it safe from using it again, based on observation of corrupted databases) and due to that last one at my most recent employer, the handy time/date/version named zip file! This was instituted by my boss, who believed in the KISS principle. Because no more than 5 of us were working on that project at one time, combined with the fact that the software architecture had no cyclic dependencies (I saw to that personally) this was actually fairly workable, considering how cheap disk space is, and how expensive time is when the database gets scrambled somehow. Long filenames are very useful, as things can often get munged when sent through utilities.

What do you think of documentation tools such as Doxygen, for instance?

I've actually downloaded and executed Doxygen on the BeOS API, and I like what it can do. Documentation tools that assist in making documentation efficiently maintained are vital to anything other than a small and simple application that is written once and never updated. I am all for such tools, as too many things are left in a state of obsolete documentation: look at the Be Book for a prime example of a documentation that would be really nice to keep perfectly updated without trying to remember what does what a long time after the fact. How many API calls are underutilized because they're undocumented? How many bugs exist within them due to the system being written in one area based on a misunderstanding of how it works in another area?

The people that should know how a system does/should work are the developers that write the code. The only documentation that can ever be completely up-to-date is within the source code, if the developer pays any attention. Any other tools that integration into the system would be desired, please, people, don't be afraid to make requests!

Will 'project X' be able to handle JAVA code, or will it be geared towards C/C++ only for the first while?

Initially, C/C++ is the focus, at least for version 1.0. Someone has already suggested Python support as a desirable item, and I will consider that as well. After all, the more people you can serve, the more sales, right?

Are collaborative features on the todo list? ie; More than one coder working (remotely) on the same project/source file?

Yes, though not as a 1.0 release item. It may be a non-version add-on later on (TBD), but the IDE design takes the thought of multiple things going on simultaneously into the threading design and data structures. I see no practical reason why people couldn't watch what others are typing as they type it, and excluding the compilation of certain source files until certain conditions have been reached. It isn't rocket science!

See you in part II!

Okay, Technix, see you on the far side of this interview ("Break on Through to the Other Side" going through my head.)