As many of the campus web developers and content owners already know, OIT made some major changes on the weekend of February 24, 2007 regarding the configuration of it's central web serving environment as it relates to how various websites access the filesystem, and persmissions that they use. We implemented a security regime based around an Apache module called mod_waklog, which was originally developed at the University of Michigan, and has been twisted by us at UMBC for our own purposes.
The changes made probably seemed drastic, heavy-handed, without planning -- however, we had been planning on making these changes over a long time frame, working with various content owners and web developers to convert their sites to the new security regime as time allowed. Due to an immediate security concern, however, we had to accelerate our time table on the site conversions, and do them all in one swoop as soon as we technically and humanly could.
(pointy hat alert -- this gets technical)
During the week or so leading up to the change, OIT's security staff had received reports from some of the campus' web developers that odd '.php' files had been showing up in their content areas. These PHP scripts contained some code which cobbled together pieces of the serving environment, and then made a PHP include() call to a URL located on a remote web server -- two actually, one in Samoa and one in Romania as a fall-back. This utilized a feature in PHP called "url_fopen", and allowed executable PHP code, or malicious content, to be processed at the time that page was accessed.
To draw people to these pages, companion .htaccess files were placed in these directories, redefining the 404 Not Found error page to point it's corresponding planted .php file. Anyone trying to visit one of these directories, but hitting an old or non-existant URL, would cause the malicious code to execute, potentially further compromising our web environment or causing the user's system to be injected with malware. Luckily, these files were easy to identify and remove from our web space before any further damage could be done. In all, we found and removed over 2,500 of these files from our web space.
However, the problem remained -- we didn't know how they got there. Over the past few years we've tried to create an open web development environment for the campus. When it started out, most if not all of the content development was done within OIT -- but as the campus' needs have grown, others have been contributing static and dynamic content -- to the tune of over 300 separate websites either hosted under www.umbc.edu, or one of our virtually hosted environments. All of these web areas, with over 800 people being identified as having "write" access to them and able to add content, run under one security domain -- which isn't so bad when you can count your web developers on one hand, but IS a problem when you've basically lost control over who's doing what.
As you can imagine, this was a problem. We were worried about the campus' image, and the integrity of our hosted sites. While we had identified changes we should make to our web environment to enhance security, and limit the exposure of one site to changes made in another, we had only rolled this out to a few select sites for testing. But the potential for exposure that apparently existed warranted immediate steps be taken, so, on the evening of Saturday, Feb 24, :
* 300+ separate security domains were created, carving off the various public/www websites that we host.
* php "misfeatures" that are popularly used to "hack" sites with, register_globals and url_fopen were disabled.
We've been working with site owners since these changes on a case-by case basis and assisting them in adapting their sites to the new security model if something seemed amiss with their functionality. Surprisingly, out of the 300+ sites that were converted, we've only had reports of a dozen or so sites that were impacted by the change.
The changes in our PHP configuration have made us immune to many of the common vectors that are used to attack PHP applications; and while there is still a chance that a web application could have a security issue with it, the damage that could be cause by it will be limited to the section of the web tree that it is designated to execute in.
We understand that these changes have caused some inconvenience, however, as you now understand, they were made with the interest of protecting the University's web presence and implementing a model that will continue to do that.