Upload progress for Firefox 3.5

The new release of CKFinder 1.4 is due mainly to cover the integration with CKEditor 3, but it includes other little details, and I'm gonna talk about one of the here: the upload graph for Firefox 3.5

 

Firefox 3.5 might look boring for the users, it doesn't bring any special change to the interface or to the general options, but under the hood there are lot of changes for developers. Old bugs fixed, polishing, new features...  And among those new features there is something special for us: the ability to read the files picked with an <input type="file"> from javascript. Previously the only option was to read the filename and then submit the form to the server, without any ability to know how long it's taking.

A little history

In order to provide a progress bar you had to use some special coding at the server, so it could provide a progress info and poll that periodically. In some context it's far easier than in others, some require proprietary components, some require just a little script, but it's not easy to get it ready for every environment.

Then the Flash uploaders appeared. Macromedia did add a nice API to flash that allowed to easily pick several files from the user computer (of course, it's the user the one that selects the files) and then upload that data to the server, with event notification so now you can find several (lots?) of libraries to do that. One example is SWFUpload. But as far as I know, all of them have a problem due to the flash plugin itself: in non-IE browsers it fails to send back to the server the proper cookies, so it ruins the ability to keep authentication in the upload process. The trick then it's to send the data for the authentication cookie along the request and then recreate the session at the server with that info, I know that the code to do that in PHP and Asp.Net it's available and you can easily add that, but I've never seen such workarounds working for Asp (I don't know about CF). And even in Asp.Net you might need to alter files (global.asax) that you can't touch "to show a progress bar".

So as you can't see there is no easy solution "that works for everyone", but in the future that can be different.

The present

Starting with Firefox 3.5 you can use a new (non-standarized) API of the XmlHttpRequest: sendAsBinary. This post at hacks.mozilla.org  does explain it much better that I would be able to do in a lifetime.

So I went ahead and coded it for CKFinder.

• Pros:
  • No changes required at the server,
  • It's based on feature detection, so if any other browser adds these APIs it will automatically work.
• Cons:
  • Only Firefox 3.5,
  • We don't know how well it will perform with big files (1).

1: The Firefox API is based on reading the full contents of a file as a string and then sending it with a XHR, so if you pick a 1Mb file it means reading 1Mb of data, that looks easy. On the other side, if you are storing a 500Mb movie, then Firefox will go ahead and use in a moment 500Mb of additional memory. Wow, looks scary so we capped the feature so it's only triggered if the file is smaller than 20Mb.

The way that it's coded means that this is transparent to the user, he doesn't have to do anything and it will work. The form is just the same, the only change is the way that the data is sent to the server.

Other options

At the same time I evaluated the ability provided by Webkit that allowed to pass a file object to the XHR.Send() method, but it has some important drawbacks:

• It's not possible to do feature detection on the ability, so it would need to be hardcoded using UserAgent strings. That's not nice
• The file is sent to the server without any of the POST boundaries to wrap the data. That means that your standard code at the server won't work, you need to write special code to send the file name (easy) and then process the boundless data at the server (not so easy). In Firefox we have to add those boundaries, but the point is: It's possible to add them, so the server doesn't notice any difference between a standard form-post and this xhr.sendAsBinary()

There are lots of talk about this kind of features for inclusion in the new versions of XHR and related APIs, so I think that it's important to start testing what's available and provide feedback about what works and what fails.

I also tested the ability that Opera and Webkit offers to select multiple files with the <input type="file"> widget, and although it worked for Webkit, Opera did send the data using a set of headers that caused Asp.Net to consider it an attack, and crashed the AspUpload ActiveX from Persits (I didn't test it further as it was a dead road).

So in the future the ability to send multiple files with webkit might be added, also we will evaluate further the option to use a flash uploader so IE users get the option to both select multiple files and get a progress status, the important part is that the adjustments at the server side should remain within the CKFinder code, when you start requesting changes to global files then the problems start

Little bugs

Nevertheless, I don't think that the Firefox implementation is bug free.

The first problem is that the progress events are fired only when the server "consumes" the data. I mean: if you look carefully or put a sleep() at the server you will notice that the upload seems to stall at the end, although the data has been sent to the server, the browser doesn't notify that, it waits there, and looks strange.

As I said, reading the full file in memory doesn't look nice, it should be possible to request sending a file and the browser would read it just like it does with a normal form POST, it can read it little by little as it needs and with minimum overhead. In that situation we wouldn't need to restrict the maximum size, and this is important because getting progress info is more interesting when the file grows bigger.

If you notice any problem with this feature, please report it so we can look at it and fix it ASAP.