Monday, October 29, 2007 1:36 PM
Sudeep Gopalakrishnan
Uploading large files in Chunks using ADODB Stream object with an Active Server Page
One of the features that some web sites provide is submission of content or files. The content may include anything from text pages to very large files containing megabytes or gigabytes of data. Usually uploading small files do not cause problems, but many developers and server administrators will tell you about problems uploading large files. As an end user you may notice this as well. This post is unlikely to make much sense to an end user unless you have a good background working with Active Server Pages.
With Internet Information Services (IIS), file uploads typically will cause the entire data coming through the request stream to be held within the IIS user mode memory before being written to the disk. On 32 bit platforms this can cause problems if a lot of people attempt to upload large amounts of data at the same time. The user mode memory space itself is restricted to 2 GB and NO process gets the entire 2 GB of virtual address space. So in this scenario, you can pretty much imagine what would happen if several users wanted to upload files as large as 1 GB+ at the same time.
Here is a sample code snippet that attempts to makes this more efficient. It uses the ADODB Stream object to binary read the file, 64 K at a time and then write it to the disk before reading the next 64 K page from the request stream.
IMPORTANT: The code sample provided below is provided "AS IS" with no warranties. My samples are not regression tested. If you do plan to use this, please test thoroughly and use it at your own risk.
<!-- Receive.asp -->
<%
dim TotalSize
dim ChunkSize
dim BytesLeft
dim CurrentBytes
dim objADO
TotalSize = Request.TotalBytes ' The total size of the file
ChunkSize = 64*1024 ' 64K pages
If ChunkSize > TotalSize Then ' If the file is smaller than 64K, then we can read this in one go
ChunkSize = TotalSize
End If
BytesLeft = TotalSize ' Number of bytes left to read is initially all bytes
Set objADO = CreateObject("ADODB.Stream")
objADO.Type = 1
objADO.Open
Do While BytesLeft > 0 ' Do until the last set of bytes has been read
If BytesLeft < ChunkSize Then ' Mostly, the last set of bytes to be read will be less than the chunk size of 64K
ChunkSize = BytesLeft ' So we modify the chunk size to the value of last set of bytes to read
End If
objADO.Write Request.BinaryRead(ChunkSize) ' Read a 64 K page or all bytes if < 64K and write bytes into a ADODB Stream.
BytesLeft = BytesLeft - ChunkSize ' Subtract the bytes left to be read from request stream.
Loop
objADO.SaveToFile "C:\WebUploads\SaveFile.txt" ' Save the stream
objADO.Close ' Close the Stream
'Dump errors.
If Err.Number <> 0 Then
Response.Write "<br>"
Response.Write err.Description
End If
%>
You could also create a simple ASP or HTM page as below to allow selecting the file you wish to upload
<!-- Upload.asp -->
<FORM NAME="myForm"
ACTION="Receive.asp"
ENCTYPE="multipart/form-data"
METHOD="post">
<INPUT TYPE="file" NAME="Upload">
<INPUT TYPE="submit" VALUE="Upload File">
</FORM>
NOTE: BTW, I owe this post to my colleague and good friend, Rahul Soni as he introduced me to Live writer. Live Writer is really cool!