File uploading thru HTTP

Introduction

File uploading thru HTTP is always big problem for websites. There are some restrictions from client and server sides. But with growing internet channels bandwidth one of major problem is file size. Sometimes it’s impossible to send 500MB file to webserver due to request length limit. One of the workaround is to increase maximal request length on webserver, but it may cause server restart when memory limit will be exceeded. For example: IIS APS.NET webserver. We increases maxRequestLength to 500MB, memoryLimit default value is 60% it means that process will be recycled when it use more than 60% of physical memory. If we have 1GB of physical memory in system and couple of users uploads simultaneously 400MB files webserver will be restarted with very high chance, because server wouldn’t have time to release memory from Requests objects.

Another big issue is file upload continuing, when process was interrupted by some reasons. Normally user needs to upload whole file once again

In this example I’ll describe how to implement file uploading method using AJAX and WebService technologies. Of course this method has its own restrictions, but it would be quite useful for intranet solutions and administrative areas in internet websites.

How it works

Main idea is quite simple. We should read file partially on send these parts to webserver.

Client Side

//Receive intial file information and init upload
function getFileParams()
{ 
 //Convert file path to appropriate format
 this.filePath = document.getElementById("file").value.replace(/\\/g, "\\\\");
 
 fso = new ActiveXObject( 'Scripting.FileSystemObject' );
 if ( !fso.FileExists(this.filePath) )
 {
  alert("Can't open file.");
  return;
 }
  
 f = fso.GetFile( this.filePath );
 this.fileSize = f.size;
 this.fileName = f.Name;
 InitStatusForm();
 InitUpload();
}

Allocate file on client, get file size. I use Scripting.FileSystemObject ActiveX object to get file size because this object will not load full file in memory.
Then init form Layout and upload process by InitStatusForm() and InitUpload functions.

function InitUpload()
{
 document.getElementById("uploadConsole").style.display = "none";
 document.getElementById("statusConsole").style.display = "block";
 
 xmlhttp = new ActiveXObject( "Microsoft.XMLHTTP" );
 xmlhttp.onreadystatechange = HandleStateChange;

 var parameters = "fileSize=" + encodeURI(this.fileSize) +
 "&fileName=" + encodeURI(this.fileName)+
 "&overwriteFile=" + encodeURI(document.getElementById("overwriteFile").checked);

 xmlhttp.open("POST","http://localhost/AJAXUpload/Upload.asmx/InitUpload", true);
 xmlhttp.setRequestHeader('Content-Type','application/x-www-form-urlencoded');
 xmlhttp.setRequestHeader("Content-length", parameters.length);
 xmlhttp.setRequestHeader("Connection", "close");
 xmlhttp.send(parameters);
}

Init upload: Create XmlHttp object and send to webservice initial information such file size, file name and overwrite flag.

//XMLHTTPRequest change state callback function
function HandleStateChange() {
 switch (xmlhttp.readyState) {
 case 4:
  response  = xmlhttp.responseXML.documentElement;
  id = response.getElementsByTagName('ID')[0].firstChild.data;
  offset = esponse.getElementsByTagName('OffSet')[0].firstChild.data;
  bufferLength = response.getElementsByTagName('BufferLength')[0].firstChild.data;
  
  percentage = (offset/this.fileSize)*100;
  if (offset<this.fileSize && !this.cancelUpload)
  {
   UpdateStatusConsole(percentage, "Uploading");
   SendFilePart(offset, bufferLength);
  }
  else
  {
   SetButtonCloseState(false);
   if (this.cancelUpload)
    UpdateStatusConsole(percentage, "Canceled");
   else
    UpdateStatusConsole(percentage, "Complete");
  }
  break;
 } 
}

Asynchronous request from server-side handled by HandledStateChange() callback function.
Parse parameters from server: id – response-request identifier, offset – start position to read file part. bufferLength – file block size to read.
If requested start position not exceed file size and upload not canceled by user we send file part.

//Read part of file and send it to webservice
function SendFilePart(offset, length)
{
 // create SOAP XML document
 var xmlSOAP = new ActiveXObject("MSXML2.DOMDocument");
 xmlSOAP.loadXML('<?xml version="1.0" encoding="utf-8"?>'+
 '<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" '+
 'xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> '+
  '<soap:Body>'+
   '<UploadData xmlns="http://tempuri.org/" >'+
    '<fileName>'+this.fileName+'</fileName>'+
    '<fileSize>'+this.fileSize+'</fileSize>'+
    '<file></file>'+
   '</UploadData>'+
  '</soap:Body>'+
 '</soap:Envelope>');
 
 // create a new node and set binary content
 var fileNode = xmlSOAP.selectSingleNode("//file");
 fileNode.dataType = "bin.base64";
 // open stream object and read source file
 if (adoStream.State != 1 )
 {
  adoStream.Type = 1;  // 1=adTypeBinary 
  adoStream.Open(); 
  adoStream.LoadFromFile(this.filePath);
 }
 
 adoStream.Position = offset;
 // store file content into XML node
 fileNode.nodeTypedValue = adoStream.Read(length);//adoStream.Read(-1); // -1=adReadAll
 if (adoStream.EOS)
 {
  //Close Stream
  adoStream.Close();
 }
 
 // send XML document to Web server
 xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");
 xmlhttp.onreadystatechange = HandleStateChange;
 xmlhttp.open("POST", "http://localhost/AJAXUpload/Upload.asmx", true);
 xmlhttp.setRequestHeader("SOAPAction", "http://tempuri.org/UploadData");
 xmlhttp.setRequestHeader("Content-Type", "text/xml; charset=utf-8");
 xmlhttp.send(xmlSOAP);
}

In this function we create XmlSoap, read file part using ADODB.Stream ActiveX object and send it to WebServer with file name and filesize.
Server response from this operation would be handled by the same HandledStateChange() callback function.

Server Side

[WebMethod]
public XmlDocument InitUpload(int fileSize, string fileName, bool overwriteFile )
{
 long offset = 0;
 string filePath = GetFilePath(fileName);
 
 if (File.Exists(filePath))
 {
  if (overwriteFile)
  {
   File.Delete(filePath);
  }
  else
  {
   using (FileStream fs = File.Open(filePath, FileMode.Append)) 
   {
    offset = fs.Length;
   }
  }
 }

 return GetXmlDocument(Guid.NewGuid(), string.Empty, offset, 
(InitialBufferLength+offset)>fileSize?(int)(fileSize-offset):InitialBufferLength);
}

Init upload server-side function. If file with such name already exist and overwrite flag is false existed file will be appended otherwise file will be deleted. Then construct response by GetXmlDocument function.

[WebMethod]
public XmlDocument UploadData(string fileName, int fileSize, byte[] file)
{
 if (fileName == null || fileName == string.Empty || file == null)
  return GetXmlDocument(Guid.NewGuid(), "Incorrect UploadData Request", 0, 0);

 string filePath = GetFilePath(fileName);

 long offset=0;
 using (FileStream fs = File.Open(filePath, FileMode.Append)) 
 {
  fs.Write(file, 0, file.Length);
  offset = fs.Length;
 }
 return GetXmlDocument(Guid.NewGuid(), string.Empty, offset, 
(InitialBufferLength+offset)>fileSize?(int)(fileSize-offset):InitialBufferLength);
}

This method handle request from client with file part data. Append file part to uploaded part and request next part.

Install and Run

To run project you should done couple manipulations:
1. Give read/write permissions to your IIS user for Upload folder.
2. Enable ActiveX objects in your IE browser. (Add website to trusted websites)

Remarks

I’ve described solution core all Layout functionality like upload panel and progress bar can be founded in project.

Please don’t use this solution AS IS in your projects. It’s just AJAX upload form example.
Working with streams, files and ActiveX objects expects handles all error cases.

Links

http://codeproject.com/Ajax/JavaScriptSOAPClient.asp - Excellent AJAX article.
http://www.15seconds.com/issue/010522.htm - File upload technique.

About Dzianis@gmail.com

Graduated in Belurassian State University on chair of Artificial Intelligence. Take a part in development video alarm system for PC with live AI analyzing multiple video streams. 3 years of ASP.NET experience in development multi-thier applications, integrated applications, enterprise level solutions. Almost 1 year experience in ASP.NET 2.0 starting from BETA 2. Now working in http://pro-softsolutions.com Click here to view Dzianis@gmail.com's online profile.