Best practice for streaming fairly large files from server to clie - DOTNET

One possible option is to redirect the browser to the file location itself.
If the file is stored in a non-http accessible folder, you could copy it
into a temp folder for download. Of course, then you have to clean it up at
some point and there is a potential security risk as other users could also
download the file.

Barring that, I think what you said would be fine. Just read chunks from a
FileStream and buffer them to the Response.

Scott


"Phil Johnson" <PhilJohnson@discussions.microsoft.com> wrote in message
news:CD97D69A-6688-47D9-A423-669B0E2B7A5E@microsoft.com...
> Hello,
>
> I have an issue with some code I am updating:
>
> An entire file is loaded into a memory stream then that memory stream uses
> its WriteTo method to write to the responses output stream. With larger
> files (250 mb) I get a system out of memory exception because the whole
> file
> is loaded into RAM.
>
> I need to modify this so the file is buffered into the output stream
> (maybe
> via a filestream?) to avoid the whole file being loaded into ram.
>
> Does anybody have any good example code or links of how to do this?
>
> Thanks in advance for any help.
> --
> Regards,
>
> Phil Johnson (MCAD)

http://groups.google.com/group/micro...13224a77b89add


See if the WriteTextFile in the HttpHelper will help you.


If it does, please post back here.

I ~think that is what you're looking for.





"Phil Johnson" <PhilJohnson@discussions.microsoft.com> wrote in message
news:CD97D69A-6688-47D9-A423-669B0E2B7A5E@microsoft.com...
> Hello,
>
> I have an issue with some code I am updating:
>
> An entire file is loaded into a memory stream then that memory stream uses
> its WriteTo method to write to the responses output stream. With larger
> files (250 mb) I get a system out of memory exception because the whole
> file
> is loaded into RAM.
>
> I need to modify this so the file is buffered into the output stream
> (maybe
> via a filestream?) to avoid the whole file being loaded into ram.
>
> Does anybody have any good example code or links of how to do this?
>
> Thanks in advance for any help.
> --
> Regards,
>
> Phil Johnson (MCAD)

Thanks for your thoughts Scott...

It looks like buffering the filestream in chunks is going to be the best
option... the files will be temporary files and I need to delete them as soon
as the data has been streamed.

--
Regards,

Phil Johnson (MCAD)


"Scott Roberts" wrote:

> One possible option is to redirect the browser to the file location itself.
> If the file is stored in a non-http accessible folder, you could copy it
> into a temp folder for download. Of course, then you have to clean it up at
> some point and there is a potential security risk as other users could also
> download the file.
>
> Barring that, I think what you said would be fine. Just read chunks from a
> FileStream and buffer them to the Response.
>
> Scott
>
>
> "Phil Johnson" <PhilJohnson@discussions.microsoft.com> wrote in message
> news:CD97D69A-6688-47D9-A423-669B0E2B7A5E@microsoft.com...
> > Hello,
> >
> > I have an issue with some code I am updating:
> >
> > An entire file is loaded into a memory stream then that memory stream uses
> > its WriteTo method to write to the responses output stream. With larger
> > files (250 mb) I get a system out of memory exception because the whole
> > file
> > is loaded into RAM.
> >
> > I need to modify this so the file is buffered into the output stream
> > (maybe
> > via a filestream?) to avoid the whole file being loaded into ram.
> >
> > Does anybody have any good example code or links of how to do this?
> >
> > Thanks in advance for any help.
> > --
> > Regards,
> >
> > Phil Johnson (MCAD)
>
>

Hi sloan, thanks for the sample.

I think I am trying to do something different though, correct me if Im wrong
though...

Your sample looks like it is a client and streams a file from a url to a
local file.

What I am trying to do is to create a server that has a file stored locally
and streams that file to the client.

--
Regards,

Phil Johnson (MCAD)


"sloan" wrote:

> http://groups.google.com/group/micro...13224a77b89add
>
>
> See if the WriteTextFile in the HttpHelper will help you.
>
>
> If it does, please post back here.
>
> I ~think that is what you're looking for.
>
>
>
>
>
> "Phil Johnson" <PhilJohnson@discussions.microsoft.com> wrote in message
> news:CD97D69A-6688-47D9-A423-669B0E2B7A5E@microsoft.com...
> > Hello,
> >
> > I have an issue with some code I am updating:
> >
> > An entire file is loaded into a memory stream then that memory stream uses
> > its WriteTo method to write to the responses output stream. With larger
> > files (250 mb) I get a system out of memory exception because the whole
> > file
> > is loaded into RAM.
> >
> > I need to modify this so the file is buffered into the output stream
> > (maybe
> > via a filestream?) to avoid the whole file being loaded into ram.
> >
> > Does anybody have any good example code or links of how to do this?
> >
> > Thanks in advance for any help.
> > --
> > Regards,
> >
> > Phil Johnson (MCAD)
>
>
>

first turn of buffering for the page. then use Response.Write in chunks,
or if its an actual file, the builtin Reponse.WriteFile
(which just read and writes chunks via .Write).


-- bruce (sqlwork.com)

Phil Johnson wrote:
> Hello,
>
> I have an issue with some code I am updating:
>
> An entire file is loaded into a memory stream then that memory stream uses
> its WriteTo method to write to the responses output stream. With larger
> files (250 mb) I get a system out of memory exception because the whole file
> is loaded into RAM.
>
> I need to modify this so the file is buffered into the output stream (maybe
> via a filestream?) to avoid the whole file being loaded into ram.
>
> Does anybody have any good example code or links of how to do this?
>
> Thanks in advance for any help.

Yeah, sorry I misread you.

...

I think the incremental Reponse.Write is what you're looking for, suggested
via another post.





"Phil Johnson" <PhilJohnson@discussions.microsoft.com> wrote in message
news:16EB4D10-C125-4519-B8E6-89ADAAF312D1@microsoft.com...
> Hi sloan, thanks for the sample.
>
> I think I am trying to do something different though, correct me if Im
> wrong
> though...
>
> Your sample looks like it is a client and streams a file from a url to a
> local file.
>
> What I am trying to do is to create a server that has a file stored
> locally
> and streams that file to the client.
>
> --
> Regards,
>
> Phil Johnson (MCAD)
>
>
> "sloan" wrote:
>
>> http://groups.google.com/group/micro...13224a77b89add
>>
>>
>> See if the WriteTextFile in the HttpHelper will help you.
>>
>>
>> If it does, please post back here.
>>
>> I ~think that is what you're looking for.
>>
>>
>>
>>
>>
>> "Phil Johnson" <PhilJohnson@discussions.microsoft.com> wrote in message
>> news:CD97D69A-6688-47D9-A423-669B0E2B7A5E@microsoft.com...
>> > Hello,
>> >
>> > I have an issue with some code I am updating:
>> >
>> > An entire file is loaded into a memory stream then that memory stream
>> > uses
>> > its WriteTo method to write to the responses output stream. With
>> > larger
>> > files (250 mb) I get a system out of memory exception because the whole
>> > file
>> > is loaded into RAM.
>> >
>> > I need to modify this so the file is buffered into the output stream
>> > (maybe
>> > via a filestream?) to avoid the whole file being loaded into ram.
>> >
>> > Does anybody have any good example code or links of how to do this?
>> >
>> > Thanks in advance for any help.
>> > --
>> > Regards,
>> >
>> > Phil Johnson (MCAD)
>>
>>
>>

I use a "double guid" solution here sometimes.

C:\stuff\

http://www.mycompany.com/stuff/


string folderName = System.Guid.NewGuid().ToString("N");
string fileName = System.Guid.NewGuid().ToString("N");


Which gives me something like this (if I'm writing out a txt file)


C:\stuff\\D680EB2590A542F7828A7F33CF705563\2DB610825821412F8C5CCFA3C642FEE2.txt
or
http://www.mycompany.com/stuff/D680E...A3C642FEE2.txt

Aka, a po' man's security model.
Its ugly, but if somebody guesses 2 guid's..... they totally rock.

...

That is if the information is sensitive.

Naturally, I have to clean up:
C:\stuff\

periodically.




"Phil Johnson" <PhilJohnson@discussions.microsoft.com> wrote in message
news:A4CA3E52-96FD-4851-82B3-5CCDA864573D@microsoft.com...
> Thanks for your thoughts Scott...
>
> It looks like buffering the filestream in chunks is going to be the best
> option... the files will be temporary files and I need to delete them as
> soon
> as the data has been streamed.
>
> --
> Regards,
>
> Phil Johnson (MCAD)
>
>
> "Scott Roberts" wrote:
>
>> One possible option is to redirect the browser to the file location
>> itself.
>> If the file is stored in a non-http accessible folder, you could copy it
>> into a temp folder for download. Of course, then you have to clean it up
>> at
>> some point and there is a potential security risk as other users could
>> also
>> download the file.
>>
>> Barring that, I think what you said would be fine. Just read chunks from
>> a
>> FileStream and buffer them to the Response.
>>
>> Scott
>>
>>
>> "Phil Johnson" <PhilJohnson@discussions.microsoft.com> wrote in message
>> news:CD97D69A-6688-47D9-A423-669B0E2B7A5E@microsoft.com...
>> > Hello,
>> >
>> > I have an issue with some code I am updating:
>> >
>> > An entire file is loaded into a memory stream then that memory stream
>> > uses
>> > its WriteTo method to write to the responses output stream. With
>> > larger
>> > files (250 mb) I get a system out of memory exception because the whole
>> > file
>> > is loaded into RAM.
>> >
>> > I need to modify this so the file is buffered into the output stream
>> > (maybe
>> > via a filestream?) to avoid the whole file being loaded into ram.
>> >
>> > Does anybody have any good example code or links of how to do this?
>> >
>> > Thanks in advance for any help.
>> > --
>> > Regards,
>> >
>> > Phil Johnson (MCAD)
>>
>>

Thanks for all the responses, they certianly got me pointed in the right
direction.... after doing a bit of looking online for the response.write in
chunks method I found this great example from Microsoft.... only requires
changing the string for the filename/path.....

http://support.microsoft.com/?kbid=812406

--
Regards,

Phil Johnson (MCAD)


"bruce barker" wrote:

> first turn of buffering for the page. then use Response.Write in chunks,
> or if its an actual file, the builtin Reponse.WriteFile
> (which just read and writes chunks via .Write).
>
>
> -- bruce (sqlwork.com)
>
> Phil Johnson wrote:
> > Hello,
> >
> > I have an issue with some code I am updating:
> >
> > An entire file is loaded into a memory stream then that memory stream uses
> > its WriteTo method to write to the responses output stream. With larger
> > files (250 mb) I get a system out of memory exception because the whole file
> > is loaded into RAM.
> >
> > I need to modify this so the file is buffered into the output stream (maybe
> > via a filestream?) to avoid the whole file being loaded into ram.
> >
> > Does anybody have any good example code or links of how to do this?
> >
> > Thanks in advance for any help.
>

Thanks for the followup post... about a concrete solution.



"Phil Johnson" <PhilJohnson@discussions.microsoft.com> wrote in message
news:6ACF1370-18EA-4BE6-B9AB-B33773257F55@microsoft.com...
> Thanks for all the responses, they certianly got me pointed in the right
> direction.... after doing a bit of looking online for the response.write
> in
> chunks method I found this great example from Microsoft.... only requires
> changing the string for the filename/path.....
>
> http://support.microsoft.com/?kbid=812406
>
> --
> Regards,
>
> Phil Johnson (MCAD)
>
>
> "bruce barker" wrote:
>
>> first turn of buffering for the page. then use Response.Write in chunks,
>> or if its an actual file, the builtin Reponse.WriteFile
>> (which just read and writes chunks via .Write).
>>
>>
>> -- bruce (sqlwork.com)
>>
>> Phil Johnson wrote:
>> > Hello,
>> >
>> > I have an issue with some code I am updating:
>> >
>> > An entire file is loaded into a memory stream then that memory stream
>> > uses
>> > its WriteTo method to write to the responses output stream. With
>> > larger
>> > files (250 mb) I get a system out of memory exception because the whole
>> > file
>> > is loaded into RAM.
>> >
>> > I need to modify this so the file is buffered into the output stream
>> > (maybe
>> > via a filestream?) to avoid the whole file being loaded into ram.
>> >
>> > Does anybody have any good example code or links of how to do this?
>> >
>> > Thanks in advance for any help.
>>