So as we can see from the specifications in the RFC document, resuming and multi-streaming download support is mostly handled by following HTTP headers: “Content-Length” and “Content-Range”. Additionally response status header “HTTP/1.1 206 Partial Content” defined for this type of requests/responses. First download request is no different from standard request. The main “magic” will begin with subsequent requests, when user (specifically the user web browser/download client) requests download resumption or the next part of multi-stream download. The users’ client sends to the web server a request with supplied position to determine where it wants to start the download. Commonly this position is equal to where the download was aborted. Also, by using this functionality, most download clients organize multi-streamed downloading through sending to the server a few requests at the same time, so multi-streamed downloading technically is the same thing as resumption of download.

After web server receives such a request, it should answer either with standard status “200″, what in fact means “download resumption is not supported” (this is how the implementation of virtual() function works), or after handling received data, server should reply with “206″ status code, content length, content range, and, of course, content itself. All this functionality is supported by default in most recent HTTP servers, but as said above, sometimes this content should be given to client through a PHP script (for example after authorization).

handle_file_download.php.tar.bz2

We stand poised on the brink of yet another paradigm shift in technology. This shift will be in the form of a simple device that most of us have a love/hate relationship with. This device I speak of is the internal Hard Drive, yes you heard me. For the last 24 years there has been no better media to store our prized data on than the spinning magneto platters that have grown from such simple beginnings as the 5MB hard disk made up of 50 two foot diameter magnetic disks or platters to today’s 1.5TB (that’s Terabytes to you and me, 1500 Gigabytes) 3.5 inch hard drive found in modern desktop and server platforms.

With a variety of interfaces that have spawned ever increasing speeds and opened up the door to new applications never dreamed of when storage required 200lbs of spinning metal to store only a few Megabytes of information, let’s not even talk about seek times. From early Bus connectors we arrived at the first standard storage connector known as IDE (Integrated Drive Electronics)and the SCSI or “scuzzy” (Small Computer System Interface) adapter was soon to follow with increased performance and longevity finally followed by the Serial adapters in the forms of SATA (Serial Advanced Technology Attachment) and SAS or Serial Attached SCSI which use the same connector types but different controllers and have become the newest standard in manufacturing and data storage connectivity.

Over the years we have seen seek times steadily dropping from “Let’s go get a cup of coffee while this file opens” measured best in minutes to the blistering 9ms (that’s milliseconds) of the modern 15k RPM SCSI drives, ear plugs not included. And with each successive increase in speed came and equally compelling desire to reduce cost and increase the speed even further until today we can purchase at any corner store or delicatessen a hard drive with 250+ GB of storage and an average seek time of sub 20ms for less than $.30 per Gigabyte.

How you ask, does this relate to some Paradigm shift?

Enter the SSD or Solid State Drive, no longer a disk, this data storage device has no moving parts and is nearly as exciting to look at as a brick. Let me quickly rewind to 2004 when I was first introduced to the world of SSD. At this time we at Silicon Valley Web Hosting were looking for a solution to a customer’s request for higher levels of data delivery. It seems that they were not happy with the SCSI RAID configuration that we had built them 2 years before and wanted to explore some other options for increasing their performance. Enter BitMicro http://www.bitmicro.com/ a pioneer in SSD technology that could deliver us a whopping 8GB of storage in the convenient form of an 80 pin 3.5″ SCSI drive for a mere $4500 each. These drives had a seek time at or below 1ms which was extraordinary even for SSD at the time and based on some preliminary testing I was able to determine that these drives would definitely relieve the bottle neck that our customer was seeing.

Alas, we could not justify the 30x increase in cost that this storage solution required and the plans for implementing SSD in our environment were shelved for another day. Fast forward to 2009 and we now see SSD being adopted on the desktop, sold in Laptops from Dell, HP, ASUS and many other manufacturers with seeming wide acceptance (and a few hiccups). Today a simple Google search for SSD returns thousands of hits for drives ranging in size and cost of 8GB for $70 to 512GB for $2000 from makers like OCZ, Transcend, Super Talent,  and here is where the shift is taking place. In the next two years the cost and reliability of these drives is going to equal that of the SATA and SAS drives however for the same money the performance and longevity of the Solid State Drives will make them the only option. Let’s take a look at the numbers really quick.

How SSD Stacks up

Today for $900 I can buy at my favorite outlet, Fry’s, Buy.com, NewEgg or wherever (though my butcher is not stocking these yet I am told they will be available in the next few weeks).

240GB Space

0.09ms seek time

Read speed of 800+MB per second

Write speed of 300+MB per second

This is $3.75 per GB which seems like an awful lot and in fact it is and awful lot in terms of storage but let’s remember that the latest SAS drives which offer similar data throughput but much longer seek times are not even half the price and these Solid State Drives are only 1 year old. The SAS drives are now ending 2 years on the  arket and the price is quite stable. I expect we will see the SSD’s continue on their path of price decay to somewhere around $.50 per GB or less. At which time the standard platter type Hard Drive will be replaced by the more energy efficient (we haven’t even talked about this yet) superior performing Solid State Drive. As Benchmark Reviews stated: “Solid State Drive offers tremendous performance in read and write bandwidth speeds and an exceptional 0.10 ms response time at a attainable price”.  Anandtech.com has this to say: “The world’s fastest consumer desktop hard drive, Western Digital’s 300GB VelociRaptor can access a random file somewhere on its platters in about 6.83ms; that’s pretty quick. Most hard drives will take closer to 8 or 9ms in this test. The Intel X25-M however? 0.11ms. The fastest SSDs can find the data you’re looking for in around 0.1ms. That’s an order of magnitude faster than the fastest hard drive on the market today.”

Is SSD for you?

Does this mean that SSD is right for you and your server applications today? For the most part my answer is going to be no. However if you have data that requires very high read times or if you are a performance junky like me then there is no substitute for the latest SSD drives from Intel and OCZ which demonstrate crippling performance numbers that leave even the fastest of conventional drives in the weeds.

Still, early adoption has its hazards and there are some bugs to be worked out with the sequential read/write actions found in a heavily loaded Database Application as well as data buffering where SSD demonstrates weakness that can result in “stuttering” or momentary freezes of the Operating System while the Drive recovers from data starvation.

When considering a hard drive choice we now have a new option that requires careful consideration or we risk the possibility of missing out on some very critical performance at a cost increase that may be a worthy burden.

To purchase, install, maintain and secure a state-of-the-art web server is associated with enormous costs. Web hosting pioneers first marketed their “server farms” for companies to simply rent space on remote servers, handling all the security and maintenance needs on behalf of their customers. However, smaller businesses didn’t need all the bandwidth and computing power available in a single computer, so the ability to rent only the amount they need on a shared server became an attractive proposition.

Ten years ago, hosting services looked to improve the shared services for the rising number of smaller businesses making use of the Internet. Since shared hosting presented too many security risks and bandwidth restrictions to be attractive to serious web developers, they adopted the latest virtualization technology from Linux developers of the Open Source movement. Virtualization systems blurred the line between the dedicated server and shared server by making a single server behave as if it were many individual servers. This way, users could “virtually” have their own private server, along with greater security and bandwidth flexibility.

Virtual Private Servers (VPS)

Shared hosting service customers access their servers files among other customers, which inherently raises security concerns. The less risky Virtual Private Servers provide private access and file storage, creating the same user experience of a dedicated server, but at significantly less cost. Virtual Private Servers are the ideal solution for web developers, ecommerce sites and other small to medium sized businesses who don’t want a dedicated server, but find shared hosting too restrictive or not powerful enough.

Each VPS runs on its own operating system, and hosting services generally include a selection of applications for online web development and testing, virtual server clustering, etc, and can allow installation of some custom applications. But users are limited to a set amount of disk space, and software restrictions are placed to prevent incompatibility with the virtualized environment. If an application in one VPS crashes, it may impact others residing on the same computer.

The Next Step in Virtualization: Virtual Dedicated Servers (VDS)

The VDS is the next step in evolution of the VPS. Like VPS on steroids, it provides users with their own a private IP address, increased security, more consistent bandwidth, more reliability and better crash protection. Because it operates within a fully isolated software environment, other accounts cannot slow down the other VDSs that share the same server with excessive CPU and bandwidth usage.

The VDS behaves more like a dedicated server by dividing one computer into more than 50 private server networks, each running an independent operating system with its own dedicated disk space. Just like a dedicated server, each VDS has an independent operating system, root access, web/mail servers, and unique web IP address. This way, application problems, security flaws and usage in one VDS, can not effect the other VDSs on the shared server computer.

The Future of Virtualization

As server hardware continues to evolve along with how we use the Internet, virtualization technology will undoubtedly continue to find new ways to tap their full potential. We can look forward to more bandwidth and capabilities beyond what we can imagine today, as we live and work in an increasingly virtual world!

By Mimi O’Connor, a contributing author for Silicon Valley Web Hosting and other technology publications