An Eventful Time - Part 1

Posted by Tom Allensworth, May 28 2009, 02:39 PM in AVSIM - General

Just over two weeks ago Black Tuesday struck and AVSIM as we knew it disappeared. As Stan, one of our tech consultants, and I watched via our secure connections to the AVSIM system that evening, we saw it crumbling before us. Connected via the phone and with silence ringing in our ears, Stan and I were speechless in disbelief. Over twelve years worth of work, tens of thousands of man hours, the contributions of thousands of AVSIM staff and FS enthusiasts, a community in every sense of the word... all gone. Or, so we thought.

There was no doubt then, and no doubt today, that AVSIM was taken out by a "hacker"�. I put hacker in quotes because I am not sure that is the correct term to use. Who would deliberately and with malice aforethought do this? More importantly, why would someone do this? But... more on that subject later.

As that evening unfolded, we thought we knew what had taken place. Since then, with the help of some world class technical support from literally around the world, we have been able to delve into the details and are now able to state unequivocally the sequence of events and ultimately, the deeds that brought AVSIM to its knees.

The hacker appeared to have attacked the PURPLE server; that is, our web and forum server first. We saw functionality start to disappear; CGI driven functions initially, and then the entire server fell silent. It was gone. We subsequently determined that he had executed a complete over-write of the PURPLE system, effectively destroying our web and forum server.

Stan and I then hopped over to GREEN, our library and email server and again saw the tracks of intrusion. Stan immediately instructed GREEN to reboot, but the damage was done. The partition table had been erased. No partition table = no boot disk = no online system.

Our immediate conclusion was that all of AVSIM had been destroyed, and that was the message that we immediately issued to our community via Nels Anderson at Flightsim.com and Francois Dumas at Simflight.com - historical competitors of AVSIM's, but staunch allies and supporters of the Flight Sim community, including the AVSIM community. The word was also sent out to all of the International Flight Simulation Consortium members, who in turn posted the word on their various web sites and forums. Thanks to the efforts of all of these fine people, by noon on Wednesday, the 13th, the world knew that AVSIM was offline, maybe for good.

We would not know for sure until we could get Stan to the Network Operations Center (NOC), where our servers were located - and that was not going to happen until the weekend of the 16th - four days after going offline.

On the morning of the 13th we were able to bring online a temporary forums system using a site that I had been experimenting with for another interest of mine. Again we put the word out via Nels, Francois and IFSC, and people started to flock into the new forum system.

What rapidly became apparent was that the flight simulation community wanted to help. Help in anyway they could; donations, technical support, forensics support, security engineering support; you name it. It was an impressive turnout by the community - people volunteering to bring their talents and knowledge into play, in order to get us back online. We were bombarded by email and forum offerings and I have to say, that outpouring of support both surprised us and helped convince me that not to bring back AVSIM was not an option and the Board of Directors emphatically agreed.

The question remained however - bring AVSIM back online in what form? With what content - did any content remain? Or would we have to start all over again? We would not know until Stan could lay his hands on the system, but we knew one thing for sure... We were coming back even if we had to start over from scratch.

GREEN and PURPLE were now "crime scenes"� as far as we were concerned. That meant that to come back, we had "bad, very bad, or worse"� choices available to us. No matter what choice we made, given as we believed that we had lost all of our backup data, we were going to be offline for a while to come:

1. We could keep the site down until both systems were forensically analyzed and the data recovered from across all installed disks, if at all possible,
2. We could purchase new servers, giving us the time to do #1, and start all over with the hope that we could recover some legacy data to repopulate the site with,
3. We could reformat and reload the OS on both servers, bring them online in a "start over"� mode, and in doing so, destroy any evidence of a hack,

We held our breath through the weekend of May 16 and 17, awaiting Stan's inspection and report out to us. On Monday morning, the 18th, Stan briefed us on the status of PURPLE and GREEN and we looked again at our options. Stan's visit allowed us to run a special tool which would hopefully allow us to re-establish the partitions on both servers. That was not to be.

PURPLE was completely gone. There was no way to recover from the partition table deletion and we did not know at that point that the hacker had over-written the entire disk, making any chance of on-site recovery impossible.

GREEN on the other hand showed some partitions that were recoverable! The first good news we had received out of this disaster. What that meant was that there was the possibility of recovering significant amounts of data as GREEN connected into our Network Accessible Server (NAS), where both servers were backed up and where the library files resided.

The realization that we might have significant amounts of data saved from the hack, the overwhelming financial support of the community, and our overwhelming need to bring AVSIM back as quickly as possible made our decision easy. On Monday afternoon the AVSIM Board of Directors unanimously voted in favor of using the community's contributions along with some of our savings to purchase two new servers. This would allow us to get back online quickly, provide for future growth, and provide the scalability, security and topology to never have to go through this again.

On Tuesday morning, the 19th, we ordered the servers along with a Gigabit Switch for delivery at the NOC on Thursday afternoon. The servers were delivered on time and Friday slowly approached.

To be continued...

Catching Up

Posted by Tom Allensworth, May 8 2009, 11:00 PM in AVSIM - General

I have not posted in a while and with so much going on, I figured it was time to put up a "status report" so to speak. A lot has happened since my last post regarding the demise of ACES. AVSIM has launched a complete system redesign, and we are in the process of getting the legacy file library back up and running. In the meantime, AVSIM (yours truly) participated in the meeting in Amsterdam on April 27th that resulted in the formation of the International Flight Simulation Consortium (IFSC). You are going to be hearing more about the IFSC shortly, so I will leave it out of this update for the time being. But, I would like to address some concerns and questions about our site upgrades and the library system. Let's see if I can "de-confuse" this a bit.

The situation with our servers and library system can get a bit complicated, and I am sure that there are a lot of confused folks out there as to what has transpired and where we are heading with our system. So let me walk through this a step or two at a time.

First, you need to understand our "topology" as the geeks like to call it; that is, our system architecture. We run three servers named purple, green and white. We used to have red and blue as well, but they have been removed and placed in the "bone yard". Purple serves up our web pages, the forum and other hosted sites. Green hosts the library and our email system as well as some other back end functions that you do not see. White has been a developmental server that has not seen too much use lately, as it is an older 32 bit machine that we are now going to put back to use.

Now, let me give you a little history. When we first considered running a file library system many years ago, there was no readily available software that could handle the load that we anticipated, nor which provided the functionality that we were seeking. A very talented member of our team, Matt Johnson, took on the task of designing and bringing online a "home grown" solution that the community has now used for years. The file library as you have known it is a true legacy of Matt's contribution to AVSIM and it has served us and the community well for a very long time.

After nearly a decade on the staff of AVSIM, Matt's real life started to take more and more of his time. Coincidentally, the library design was getting a bit long in the tooth and in need of updating, both within the front and back end - what you see and what takes place behind the scenes when you want to get that download. But, without a replacement for Matt, we were pretty much in limbo. And that's the way things stayed for the better part of the fall of 2008 and first four months of 2009.

Suddenly, the server we hosted the library on (Green) started showing signs of age and issues with disks. In the midst of these issues we found that numerous hack attempts were being made on Green and that the frequency of the attempts were increasing. As a result of all these issues, we took the decision last weekend to shut the library down. That would give our new tech team the opportunity to look at the hardware, explore the back end, and get a fix or fixes in place. Our goal is simply to get the legacy library system back online and then to start work on a redesign.

Prior to the failure of Green, we had decided to do a "remake" of AVSIM. Our priorities are straightforward; first, redesign the library user interface, clean up the back end, add functionality and clean up some of the functions that are already there, and make sure that our implementation of the new library system is as fast and reliable as it can be. The next priority is to redesign the AVSIM Web site and news service.

So, the legacy library remains offline as of this writing and our tech team is working to bring it back. Once that is done, we start efforts on the new AVSIM. I am looking forward to the change and hope you are too.

The Demise of ACES Studio - Part 2

Posted by Tom Allensworth, Jan 23 2009, 06:42 PM in Tom's Editorials

As the story of the demise of ACES Studios continues to unfold, here is some of what we do know to be fact, followed by opinion and conjecture.

Yesterday, Thursday the 22^nd, from 0630 to 0730 PST, Microsoft's CEO, Steve Ballmar's email regarding layoffs appeared in employee mailboxes throughout the company.

At 0730, the ACES team was informed via email of an "All Hands" meeting to be held at 1100.

At the All Hands meeting, team members were informed that all but six members are being let go. The six who remain will be charged with archiving the code and supporting some existing contracts. That effort will last for approximately six months.

MS Human Resources sent out two different emails to the unfortunate 1,400 employees selected for layoff. The first of these informed the employee that Friday, January the 23^rd, would be their last day of employment and that at the end of the day, their badge and email would be non-functional. The second email was to employees that were given somewhat of a reprieve. This email informed the more fortunate MS employees that they would have 60 days to find new positions within Microsoft. They could continue to use their email, badge and offices, but at the end of 60 days, they could not. They have until March 23^rd to find new employment within MS or they are out the door. How the two emails were distributed throughout the ACES team we are not able to determine - we are trying to obtain insight into that split.

There are rumors and conjecture circulating that MSFS will continue in some form, including this article published on IGN that quotes a member of the MS' public relations firm. Our sources confirm that there has been some discussion regarding the future of MSFS. However, those discussions were very generic, and no firm way forward has been decided upon.

We are told by multiple sources that any effort on an interim MSFS "holding" effort won't start until the RIF's (Reductions In Force – or in layman's terms, layoffs) have concluded. Contrary to the community's understanding, the ESP and FS teams are two separate entities. The holding effort's project manager will most likely be a member of the ESP team.

Of concern is the state of the "code base" of MSFS. Since a new rendering engine was under development by the studio, the issues in DX9 and DX10 were left by the wayside awaiting the outcome of the new engine development. With the demise of the FS team, development of the new rendering engine comes to a halt. We are left with the flaws and short comings in DX9 and DX10. Of greater concern is that the talent working on the rendering engine and knowledgeable of the flaws in DX9 and 10 are now gone. We seriously question whether the MS Management team realizes this and its implications.

Speculation in the flight simulation community has been rife ever since the introduction of the Xbox Console that MSFS was fighting an uphill battle as a PC based product (and implicitly there was an uphill struggle for support of the PC based FS by MS Management). There are rumors that this RIF of the ACES Studio affirms a movement for Microsoft Games Studios to morph into the Xbox Games Studio and without senior MS Management support, that indeed, morphing will take place.

Cutting to the chase… In the absence of an ACES Studio and zero future development or support of the FS franchise, will our hobby survive? In an earlier conversation this morning with Alex Ford of JustFlight of the UK, this question was pondered. Alex's fundamental position is that where adversity exists, opportunity lies. Alex stated that in the absence of any further development of the core FSX product that tremendous opportunities continued to exist for developers to create add-on's for the product. As others in the forums have pointed out, a four or five year stable product base is a wonderful opportunity to maximize the potential of FSX. We at AVSIM agree with this sentiment and we'll support any and all efforts to make it so.

The Demise of ACES Studio - Part 1

Posted by Tom Allensworth, Jan 23 2009, 11:40 AM in Tom's Editorials

This morning, Friday, January 23^rd, we woke up to a new reality. To the extent to which we know what that new reality is, that is. What do we know? We know that Microsoft announced yesterday morning that some 5,000 people would be laid off in the coming months; 1,400 of them to be laid off immediately. An unprecedented event – a layoff has never before occurred at Microsoft. No indication of which business sectors of Microsoft were provided. By 5 p.m. east coast time in the U.S., it had become painfully apparent where some of those lay off's were taking place.

By Thursday afternoon, games sites were reporting the rumors of the complete demise of the ACES Team. That is, Microsoft had shut the entire team down and work came to a screeching halt on their two products of most interest to this readership; Flight Simulator and Train Simulator. Phil Taylor, a former member of the ACES Team and now employed by INTEL posted a note on the AVSIM Forums Thursday evening and added significant credibility to the earlier posts on other game sites. Since his post, the messages have been rolling into the forums and the readership from around the world is hovering, attempting to divine "what next".

To say that the collective flight simulation community is in a state of shock is probably this year's greatest understatement. And, understandably, there is a tremendous amount of hand wringing going on, interspersed with some very insightful commentary on the future of the hobby and the genre of flight simulation.

As I write this, the AVSIM staff is attempting to get more detail on how extensive the layoff at ACES was, what insights might be possible into the future of the MSFS franchise, and where some of the experts from the MS ACES team see our corner of the games genre heading, given this new reality. Keeping in mind that the ACES Team was and probably still is under a Non Disclosure Agreement (NDA), it is unlikely that we will be able to do any attributable interviews. We also want to talk with some of the larger third party developers and get their input and thoughts on the future.

We will post more later as we can bring facts to light...

A New Super Library?

Posted by Tom Allensworth, Jan 1 2009, 11:35 AM in AVSIM - General

This was posted earlier today as forum news. Have a read and let me know what you think!

What if… What if a super Library System were available that could do this:

• Downloading:
• · The most powerful search functions available, including "sounds like" searches using SoundEx technology
• · View number of downloads
• · View revision details
• · View license details
• · View file Update detail
• · Subscribe via RSS to entire library or single category (that's right; if a file is uploaded to a category, as an example - FSX Original Aircraft, you will receive an RSS notice with those uploads shown)
• · Automatic email to user of updates to file – if an author updates a file or creates a revision, you will receive immediate notification
• · Ability to comment and rate the file
• · One click "Find all files by this member"
• · Add files to your favorites within your Control Panel
• · Send the link to the file via email to a friend
• · File upload notification automatically posted in a special forum, where you can get support on the download from your fellow forum members and possibly the author himself
• · Control over number of files shown per page
• · View the last 10 submissions or view all submissions since your last visit to the library
• · View all additional information provided by author; i.e., links to resource material, real world images, etc.

Authors and Uploaders:

• · No more multiple pages to upload; a single page with all the necessary information blocks provided for you
• · No more strict file name requirements; names can be descriptive, i.e.,
  • o MOONEY CADET B210 Version 1.0
  • o B777-200 Air Pacific PW Version 2.1
• · File descriptions can use BBCode as well as providing links to online resources, documentation, etc.
• · Revision feature allows you to upload file and provide revision notes. No more multiple file uploads with lengthy notes and URL's for earlier versions
• · Support to downloaders via your own web site, your email, or via the AVSIM Forums
• · When you file is approved, it automatically creates a message in the File Library Content forum where you can provide additional information, hear the feedback of the downloaders and view "squawks" for future fixes / revisions.
• · No more thumbnail requirements! Upload one image (up to 200K in size) and the system creates a thumbnail for you and places it next to your file listing.
• · No more convoluted, multi-page upload process – one page and you are there!

Merry Christmas Everyone!

Posted by Tom Allensworth, Dec 25 2008, 02:28 PM in AVSIM - General

We would like to wish everyone a Merry Christmas! May your day be blessed with closeness, warmth and love of family and friends!

As a Christmas gift to all members of the AVSIM Forums system, I have posted some of my collection of digital aviation art. There are 171 images in this album, and I encourage you to look through these images. There are some very famous art work in this collection - or at least famous amongst us aviation buffs!

You can view the album here.

Here's just one entitled "So Lucky Again" - the launch sequence for Jimmy Dolittle's raid on Tokyo in early 1942 with B-25 Mitchells:

Is Avsim Broke, Again?

Posted by Tom Allensworth, Dec 18 2008, 10:19 AM in Tom's Editorials

I have to admit that I am continuously surprised by the lack of computer and Internet savvy that some flight simmers exhibit. Consider this a "rant" if you like or if you would prefer a rare editorial from yours truly.

I had thought that everyone on the planet who communicates with something more sophisticated than two cans and a string had heard about the critical flaw in the Internet Explorer browser. Everyone, or so I thought, knew that a patch was forthcoming in an "out of band" release from Microsoft to address this possible exploit. Two fundamentally flawed assumptions.

The third assumption that appears to be flawed is that people would anticipate changes to their browser and immediately suspect the patch if their browser behavior suddenly changed. Three strikes and I am out (Interesting! Microsoft Word wants to change that to read "Three strikes and I ARE out… Maybe I should just let it have its way with me).

And I guess that is the point; when we turn on auto downloading of patches and updates, we ARE letting Microsoft have their way with us. And that is not necessarily a bad thing. I allow MS to do its thing on both of my Vista 64 based systems; I don't have the time to worry about checking and fiddling with updates and patches. If MS screws up a patch or an update, I can always change the systems back to the last known good configuration.

I know that during the evening of the second Tuesday of every month, MS will issue whatever patches and updates are completed and ready for prime time. I also know that on the morning of the second Wednesday of every month that if I fire up my browser and find issues with AVSIM, CNN, the BBC or my bank's site, then it is nearly a 100% surety that the issue results from a problem caused by the patch or update.

I will never forget the infamous "in frame security patch", and that story illustrates my point perfectly. AVSIM, since its inception, has run our main web page using a frame system; that is, we have the upper menu and banner ad area, and we have the lower framed "main page". At some point, Microsoft determined that "In-Frame" web pages could be a security risk and in an update to IE, forced the security switch to "don't trust in-frame pages". Guess what? The staff of AVSIM was bombarded by email from hundreds of readers who just knew that our site was broke (and, by the way, we had to fix it ASAP otherwise the earth would come to an end).

Last night MS released it's out of band patch for the security issues with IE. I can hardly wait to see what impact that patch has had on our readership and how AVSIM is broke.

A Story About Mike's P-51

Posted by Tom Allensworth, Dec 18 2008, 08:58 AM in Friends and Family

If you have taken a stroll through my albums within the AVSIM Gallery System, you will have come across some images of a P-51D Mustang, which are shown above in thumbnail form. A friend of mine, Mike, and his partner, Peter, own this beautiful World War II vintage aircraft, and I had the privelage of visiting her in 2006 at her new home in Adelaide, Australia. It was just coincident that I arrived in Adelaide the same day that she was flown from her original location, Melbourne. These photos were taken on a Monday, and both she and I had arrived on the previous day. She from Melbourne, me from the U.S.

She was a beautiful site to behold. If you are an aviation history enthusiast, the greatest thrill you can experience is being able to exlpore a war bird, and one of the most famous at that. If you have ever heard a P-51 in flight, you have experienced the deep throated roar of one of man's finest achievements. The P-51 is truly a work of art, and the history of her development, trials and entry to war are equally amazing. I suggest you read a fairly comphrensive history of this famous warbird at the Wikipedia.

As you will read there, the Royal Australian Air Force (RAAF), Number 3 Squadron, first adopted the P-51 in Italy in 1944, replacing their P-40 Warhawks with this aircraft. The RAAF ended up purchasing 500 Mustangs. Uniquely, the Australian P-51's were produced in Australia by CAC, the only non-U.S. produced Mustangs through the life of it's production. The "Flying Undertaker" was one of these Australian built Mustangs. Someday, I will have to get Mike to send me a bit of her service history so that we can complete the story here.

Shortly after I visited her, a disaster unfolded. On a clear day in Adelaide the Flying Undertaker was taken up for a spin around the country side. Upon touch down back at her home airport, her port main gear collapsed. Pilot unharmed, the same could not be said for the Undertaker. Left wing tip, Props and of course the port side main gear were all damaged. In any other aircraft a prop strike could cause a crankshaft to shatter, main bearings to be ruined, and lord only knows what else. In their wisdom, the designers of the Rolls Royce anticipated prop strikes and built the engine to handle it.

Some day soon Mike and Peter hope to see the Flying Undertaker take flight again. I hope I am there when she does.

Would You Like To Own An Avsim Blog?

Posted by Tom Allensworth, Dec 16 2008, 08:50 AM in AVSIM - General

As we brought the AVSIM Blogs online over this last weekend, I was personally excited about the possibilities. Yes, there are millions of blogs out there, but how many are hosted by a service that is focused specifically on your hobby? How many blog services provide you immediate exposure to over 50,000 of your fellow hobbyists? None that I am aware of.

And thus, my excitement. Sure, I have set up a blog or two over the last couple of years but they could never hold my attention too long. They withered on the vine due to the recognition that my blog was one among millions, and really not located where I wanted it to be; in the middle of my hobby! The result? The AVSIM Blogs.

Our plans for supporting you, the AVSIM Blogger, are pretty straight forward too. We are going to provide wide and intensive coverage of our "blogosphere" members and give your story (or stories) the attention they deserve. All you need do is provide that story.

What are we looking for from our bloggers in terms of content? Anything aviation related. Your personal aviation stories, your flight simming experiences, learning to fly, flying how to's, instructional input, airports you have flown into, aviation travel logs, and so on. We do not want your blog to be so narrow in focus that you struggle to put up material, and we do encourage you to take a wider view of our hobby and your aviation interests in considering content for you blog.

If you are interested in publishing your own AVSIM Blog and you meet the criteria set out in an earlier blog entry on the subject, you are most welcome to join our growing flight simulation "blogosphere". If interested, send me an email or a PM and we'll get you blogging in no time.

Screen Shot Forum News And Update

Posted by Tom Allensworth, Dec 14 2008, 04:26 PM in AVSIM - Policies

Today, we posted the following message in the screen shot forum, regarding the future of the forum and the future use of the Gallery: