Is it possible to create a file that never completes its download process?

Question

Is it possible to built a file from scratch that when it gets downloaded via HTTP, the download never actually completes ? I am not talking about ZIP BOMB here.

Some download software allows you to download streaming events, thus the final size of the file is not known. Is it possible to craft a file where the download software is not able to "guess" the actual file size and keeps on downloading?

The download software never "guesses" the file size. When a file transfer is initiated the server sends some header information and in that header info the download software is told how big the file will be before it starts downloading. It is quite easy to misconfigure a server to not announce the file's size and simply send a file to be downloaded so your download software will just keep downloading until the server stops sending. — MonkeyZeus, Mar 07 '17 at 13:37
Could you draw a line between the definition of download and stream? A file download is, to me, finite; while a stream runs until I tell it to stop. I can't tell if you're mincing terms on accident. — zero298, Mar 07 '17 at 13:44
@MonkeyZeus: A server is not necessarily misconfigured if it can't tell how large the response is. It might literally not know, that's why there is Transfer-Encoding: chunked. (You could argue that the server could just buffer the output from CGIs, but let's not.) — Oskar Skog, Mar 07 '17 at 13:44
@OskarSkog True, but my main point still stands. The download software doesn't guess, it gets told. — MonkeyZeus, Mar 07 '17 at 15:05
@MonkeyZeus At least firefox seems to only treat the announced size as hint. When the server gracefully closes the connection and less than the announced size has been downloaded it considers the download successful. (That's very annoying when using SOCKS proxies, since those turn connection loss into gracefully closed connections, resulting in silently truncated downloads) — CodesInChaos, Mar 07 '17 at 15:45
This question is incredibly vague. You're probably not talking about "creating a file", but it's hard to tell what you actually mean. — pipe, Mar 07 '17 at 17:33
So many different ways to do this, the question is actually poor. A file is simply an arbitrary chunk of data which could easily be concurrently modified to produce your desired result. — Tim Hallman, Mar 07 '17 at 19:37
I'm missing something. What does this have to do with information security? — Paul Draper, Mar 07 '17 at 19:52
@PaulDraper the concept can be used as a denial-of-service: keeping a connection open and forcing a client to fill its memory/disk with junk. — adelphus, Mar 07 '17 at 21:43

Oskar Skog · Accepted Answer · 2017-03-07T11:49:27.010

29

Yes it is possible. You just need to use the chunked transfer encoding. https://en.wikipedia.org/wiki/Chunked_transfer_encoding

Depending on your server's configuration, you might be able to simply create a CGI script that writes and flushes stdout in an infinite loop.

It does not seem to work on Lighttpd which I believe buffers the entire output from the CGI script before sending it to the client. It might work on other webservers though.

Example:

HTTP/1.1 200 OK\r\n
Transfer-Encoding: chunked\r\n
Content-Type: text/plain\r\n
\r\n
1e\r\n
Uh-oh, this will never stop.\n
1e\r\n
Uh-oh, this will never stop.\n

followed by an infinite repetition of "1e\r\nUh-oh, this will never stop.\n"

edited Mar 07 '17 at 11:49

answered Mar 07 '17 at 09:45

Oskar Skog

565
6
14

1

Do you really need the chunked transfer encoding? Why wouldn't sending lots of data (and throttling it, if desired) not be possible without chunks? – CL. Mar 07 '17 at 10:52
18

@CL. otherwise the browser expects to get a Content-Length header and will discard any data beyond that length, so it can't be infinite. – André Borie Mar 07 '17 at 11:05
5

AFAIK browsers do not actually expect it in all cases: Is the Content-Length header required for a HTTP/1.0 response? – CL. Mar 07 '17 at 11:45
6

With HTTP/1.0 the request ends with eof OR the length defined by the optional content-length header. With HTTP/1.1 both options still exist and additionally there is chunked transfer encoding. – Steffen Ullrich Mar 07 '17 at 11:48
7

HTTP 1.1 says that Content-Length "SHOULD be sent whenever the message's length can be determined prior to being transferred". – CL. Mar 07 '17 at 11:50

score 15 · Answer 2 · answered Mar 07 '17 at 09:47

15

Yes, just stream /dev/urandom to the client. First, maybe you'll need to fake the file header, so that client thinks it's downloading the stuff it requested, and after that just stream random junk.

An idea on how to do this in Python:

with open("/dev/random", 'rb') as f:
    print repr(f.read(10))

answered Mar 07 '17 at 09:47

roman

323
2
6

19

Also don't forget to actually use urandom in the code so it won't block while refilling the entropy pool. – J.A.K. Mar 07 '17 at 11:52
2

@OskarSkog: Manual flushing is not necessary, the buffer is not very large and will automatically get flushed when it fills. – Dietrich Epp Mar 07 '17 at 17:40
5

If all you're trying is a basic DOS, just use zeroes. – chrylis -cautiouslyoptimistic- Mar 07 '17 at 21:58

score 5 · Answer 3 · answered Mar 07 '17 at 21:14

If the output is dynamically generated by the server software, it is possible to create a stream that keeps going on until you break the connection. However if you literally want a file it cannot be infinite in size.

However if your file system supports sparse files you can create a file that is larger than the storage media and that way produce a file which would take such a long time to download that it isn't feasible to download it all.

The maximum file size differs between file systems. On ext4 the limit is 16TB. On tmpfs the limit is 8EB. Here is a couple of examples on how such files could be created:

dd if=/dev/null of=/dev/shm/sparse bs=1 seek=7E
dd if=/dev/null of=/tmp/sparse bs=1 seek=15T

Beware when putting such files on a webserver. If the server software you are using doesn't throttle the bandwidth it is possible for a malicious client to overload your network.

A way to create sparse files that is easier to remember: truncate -s15T /tmp/sparse — b0fh, Mar 07 '17 at 22:37

Is it possible to create a file that never completes its download process?

3 Answers3