How To Upload Files To Your Facebook

A common feature in web applications is to let users upload files to the server. The HTTP protocol documents the mechanism for a client to upload a file in RFC 1867, and our favorite web framework Flask fully supports it, but at that place are many implementation details that autumn outside of the formal specification that are unclear for many developers. Things such as where to store uploaded files, how to utilise them afterwards, or how to protect the server against malicious file uploads generate a lot of confusion and dubiousness.

In this article I'thou going to show you how to implement a robust file upload feature for your Flask server that is compatible with the standard file upload support in your spider web browser as well as the absurd JavaScript-based upload widgets:

A Basic File Upload Grade

From a high-level perspective, a client uploading a file is treated the same as any other form data submission. In other words, yous have to define an HTML form with a file field in it.

Here is a unproblematic HTML page with a class that accepts a file:

          <!doctype html> <html>   <head>     <title>File Upload</championship>   </caput>   <body>     <h1>File Upload</h1>     <grade method="POST" activity="" enctype="multipart/form-data">       <p><input type="file" name="file"></p>       <p><input type="submit" value="Submit"></p>     </class>   </body> </html>

As you probably know, the method attribute of the <form> chemical element can be GET or Postal service. With Go, the data is submitted in the query string of the request URL, while with Mail service it goes in the asking body. When files are being included in the form, you must employ Postal service, as it would be impossible to submit file information in the query string.

The enctype attribute in the <class> element is commonly not included with forms that don't take files. This aspect defines how the browser should format the data before it is submitted to the server. The HTML specification defines three possible values for it:

application/x-www-form-urlencoded: This is the default, and the all-time format for any forms except those that contain file fields.
multipart/form-information: This format is required when at to the lowest degree 1 of the fields in the form is a file field.
text/patently: This format has no practical apply, so you lot should ignore it.

The actual file field is the standard <input> element that we use for nigh other form fields, with the blazon set to file. In the example in a higher place I haven't included any additional attributes, but the file field supports two that are sometimes useful:

multiple can be used to permit multiple files to exist uploaded in a single file field. Instance:

                      <input blazon="file" proper name="file" multiple>

accept tin can be used to filter the allowed file types that tin exist selected, either by file extension or by media type. Examples:

                      <input type="file" proper noun="doc_file" accept=".md,.docx">     <input type="file" name="image_file" accept="image/*">

Accepting File Submissions with Flask

For regular forms, Flask provides access to submitted class fields in the asking.form lexicon. File fields, still, are included in the asking.files dictionary. The asking.form and asking.files dictionaries are actually "multi-dicts", a specialized dictionary implementation that supports duplicate keys. This is necessary because forms can include multiple fields with the same name, every bit is frequently the instance with groups of check boxes. This also happens with file fields that permit multiple files.

Ignoring of import aspects such as validation and security for the moment, the short Flask application shown below accepts a file uploaded with the form shown in the previous section, and writes the submitted file to the current directory:

          from flask import Flask, render_template, request, redirect, url_for  app = Flask(__name__)  @app.route('/') def alphabetize():     return render_template('alphabetize.html')  @app.route('/', methods=['POST']) def upload_file():     uploaded_file = request.files['file']     if uploaded_file.filename != '':         uploaded_file.relieve(uploaded_file.filename)     return redirect(url_for('index'))

The upload_file() function is decorated with @app.route so that it is invoked when the browser sends a Post asking. Note how the same root URL is split up between two view functions, with index() gear up to accept the Get requests and upload_file() the POST ones.

The uploaded_file variable holds the submitted file object. This is an instance of class FileStorage, which Flask imports from Werkzeug.

The filename aspect in the FileStorage provides the filename submitted by the client. If the user submits the course without selecting a file in the file field, then the filename is going to be an empty string, and so it is important to e'er check the filename to make up one's mind if a file is available or not.

When Flask receives a file submission it does not automatically write information technology to deejay. This is actually a good thing, because it gives the application the opportunity to review and validate the file submission, every bit you will see afterward. The actual file information can be accessed from the stream attribute. If the application just wants to salvage the file to disk, then it can call the save() method, passing the desired path as an argument. If the file's save() method is not called, then the file is discarded.

Desire to test file uploads with this application? Make a directory for your awarding and write the code in a higher place as app.py. Then create a templates subdirectory, and write the HTML folio from the previous section every bit templates/index.html. Create a virtual environment and install Flask on it, then run the application with flask run. Every time you submit a file, the server volition write a copy of it in the current directory.

Earlier I move on to the topic of security, I'g going to discuss a few variations on the lawmaking shown in a higher place that you lot may find useful. As I mentioned before, the file upload field tin exist configured to take multiple files. If you lot use request.files['file'] equally above yous will become only one of the submitted files, but with the getlist() method you can access all of them in a for-loop:

                      for uploaded_file in request.files.getlist('file'):         if uploaded_file.filename != '':             uploaded_file.save(uploaded_file.filename)

Many people code their grade treatment routes in Flask using a unmarried view part for both the GET and POST requests. A version of the instance application using a unmarried view function could be coded as follows:

          @app.road('/', methods=['Become', 'POST']) def index():     if request.method == 'Post':         uploaded_file = request.files['file']         if uploaded_file.filename != '':             uploaded_file.relieve(uploaded_file.filename)         render redirect(url_for('alphabetize'))     return render_template('index.html')

Finally, if y'all employ the Flask-WTF extension to handle your forms, yous can use the FileField object for your file uploads. The grade used in the examples you lot've seen and so far can be written using Flask-WTF as follows:

          from flask_wtf import FlaskForm from flask_wtf.file import FileField from wtforms import SubmitField  class MyForm(FlaskForm):     file = FileField('File')     submit = SubmitField('Submit')

Annotation that the FileField object comes from the flask_wtf package, unlike most other field classes, which are imported direct from the wtforms parcel. Flask-WTF provides two validators for file fields, FileRequired, which performs a cheque similar to the empty string cheque, and FileAllowed, which ensures the file extension is included in an allowed extensions list.

When you employ a Flask-WTF course, the data attribute of the file field object points to the FileStorage instance, and so saving a file to deejay works in the same way as in the examples above.

Securing file uploads

The file upload example presented in the previous section is an extremely simplistic implementation that is not very robust. One of the almost of import rules in spider web development is that data submitted past clients should never be trusted, and for that reason when working with regular forms, an extension such equally Flask-WTF performs strict validation of all fields before the form is accepted and the data incorporated into the application. For forms that include file fields at that place needs to be validation as well, considering without file validation the server leaves the door open to attacks. For example:

An aggressor tin upload a file that is so large that the disk space in the server is completely filled, causing the server to malfunction.
An aggressor can craft an upload request that uses a filename such as ../../../.bashrc or similar, with the attempt to fox the server into rewriting system configuration files.
An assailant can upload files with viruses or other types of malware in a place where the application, for example, expects images.

Limiting the size of uploaded files

To prevent clients from uploading very big files, you can use a configuration option provided by Flask. The MAX_CONTENT_LENGTH option controls the maximum size a request torso tin have. While this isn't an option that is specific to file uploads, setting a maximum asking torso size effectively makes Flask discard whatever incoming requests that are larger than the allowed amount with a 413 status lawmaking.

Let's modify the app.py instance from the previous section to just have requests that are upwards to 1MB in size:

          app.config['MAX_CONTENT_LENGTH'] = 1024 * 1024

If you try to upload a file that is larger than 1MB, the awarding will at present refuse information technology.

Validating filenames

We can't really trust that the filenames provided by the customer are valid and condom to apply, so filenames coming with uploaded files have to be validated.

A very elementary validation to perform is to make sure that the file extension is 1 that the application is willing to accept, which is similar to what the FileAllowed validator does when using Flask-WTF. Let's say the application accepts images, then it can configure the list of approved file extensions:

          app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif']

For every uploaded file, the awarding can brand sure that the file extension is i of the allowed ones:

                      filename = uploaded_file.filename     if filename != '':         file_ext = os.path.splitext(filename)[one]         if file_ext not in current_app.config['UPLOAD_EXTENSIONS']:             abort(400)

With this logic, any filenames that do not accept i of the approved file extensions is going to be responded with a 400 error.

In addition to the file extension, information technology is also important to validate the filename, and any path given with information technology. If your application does non intendance about the filename provided past the client, the most secure style to handle the upload is to ignore the client provided filename and generate your own filename instead, that y'all pass to the save() method. An example use case where this technique works well is with avatar epitome uploads. Each user's avatar tin can exist saved with the user id as filename, and so the filename provided by the client can exist discarded. If your application uses Flask-Login, you could implement the following salvage() call:

          uploaded_file.save(os.path.join('static/avatars', current_user.get_id()))

In other cases it may be improve to preserve the filenames provided by the client, so the filename must exist sanitized showtime. For those cases Werkzeug provides the secure_filename() part. Let'south see how this function works by running a few tests in a Python session:

          >>> from werkzeug.utils import secure_filename >>> secure_filename('foo.jpg') 'foo.jpg' >>> secure_filename('/some/path/foo.jpg') 'some_path_foo.jpg' >>> secure_filename('../../../.bashrc') 'bashrc'

Equally you encounter in the examples, no matter how complicated or malicious the filename is, the secure_filename() function reduces it to a flat filename.

Permit's contain secure_filename() into the case upload server, and too add a configuration variable that defines a dedicated location for file uploads. Here is the complete app.py source file with secure filenames:

          import os from flask import Flask, render_template, asking, redirect, url_for, abort from werkzeug.utils import secure_filename  app = Flask(__name__) app.config['MAX_CONTENT_LENGTH'] = 1024 * 1024 app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] app.config['UPLOAD_PATH'] = 'uploads'  @app.route('/') def alphabetize():     render render_template('index.html')  @app.route('/', methods=['Postal service']) def upload_files():     uploaded_file = asking.files['file']     filename = secure_filename(uploaded_file.filename)     if filename != '':         file_ext = os.path.splitext(filename)[1]         if file_ext not in app.config['UPLOAD_EXTENSIONS']:             abort(400)         uploaded_file.save(os.path.join(app.config['UPLOAD_PATH'], filename))     return redirect(url_for('index'))

Validating file contents

The third layer of validation that I'm going to discuss is the most complex. If your awarding accepts uploads of a certain file type, it should ideally perform some form of content validation and reject whatsoever files that are of a different type.

How you accomplish content validation largely depends on the file types your application accepts. For the example awarding in this article I'm using images, and so I can utilise the imghdr parcel from the Python standard library to validate that the header of the file is, in fact, an epitome.

Let'south write a validate_image() part that performs content validation on images:

          import imghdr  def validate_image(stream):     header = stream.read(512)     stream.seek(0)     format = imghdr.what(None, header)     if non format:         return None     return '.' + (format if format != 'jpeg' else 'jpg')

This function takes a byte stream as an argument. Information technology starts by reading 512 bytes from the stream, and then resetting the stream pointer back, considering later when the salve() function is called we want information technology to see the entire stream. The first 512 bytes of the epitome data are going to be sufficient to identify the format of the prototype.

The imghdr.what() function can expect at a file stored on disk if the offset argument is the filename, or else it tin can look at information stored in retentivity if the start statement is None and the information is passed in the second argument. The FileStorage object gives united states a stream, so the about convenient option is to read a rubber amount of data from it and laissez passer information technology as a byte sequence in the second argument.

The return value of imghdr.what() is the detected paradigm format. The office supports a diverseness of formats, amidst them the pop jpeg, png and gif. If not known image format is detected, then the return value is None. If a format is detected, the name of the format is returned. The most user-friendly is to render the format every bit a file extension, considering the application tin can then ensure that the detected extension matches the file extension, so the validate_image() role converts the detected format into a file extension. This is as simple as adding a dot as prefix for all image formats except jpeg, which normally uses the .jpg extension, so this instance is treated every bit an exception.

Here is the complete app.py, with all the features from the previous sections plus content validation:

          import imghdr import os from flask import Flask, render_template, asking, redirect, url_for, abort from werkzeug.utils import secure_filename  app = Flask(__name__) app.config['MAX_CONTENT_LENGTH'] = 1024 * 1024 app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] app.config['UPLOAD_PATH'] = 'uploads'  def validate_image(stream):     header = stream.read(512)     stream.seek(0)      format = imghdr.what(None, header)     if non format:         render None     return '.' + (format if format != 'jpeg' else 'jpg')  @app.route('/') def alphabetize():     return render_template('index.html')  @app.route('/', methods=['Mail']) def upload_files():     uploaded_file = request.files['file']     filename = secure_filename(uploaded_file.filename)     if filename != '':         file_ext = os.path.splitext(filename)[1]         if file_ext not in app.config['UPLOAD_EXTENSIONS'] or \                 file_ext != validate_image(uploaded_file.stream):             abort(400)         uploaded_file.save(os.path.join(app.config['UPLOAD_PATH'], filename))     return redirect(url_for('index'))

The just modify in the view office to incorporate this last validation logic is here:

                      if file_ext not in app.config['UPLOAD_EXTENSIONS'] or \                 file_ext != validate_image(uploaded_file.stream):             abort(400)

This expanded bank check first makes sure that the file extension is in the allowed listing, and and so ensures that the detected file extension from looking at the data stream is the same as the file extension.

Before you test this version of the awarding create a directory named uploads (or the path that you lot divers in the UPLOAD_PATH configuration variable, if different) so that files tin can be saved there.

Using Uploaded Files

Yous at present know how to handle file uploads. For some applications this is all that is needed, every bit the files are used for some internal process. But for a large number of applications, in particular those with social features such equally avatars, the files that are uploaded by users have to exist integrated with the application. Using the example of avatars, once a user uploads their avatar image, whatsoever mention of the username requires the uploaded prototype to appear to the side.

I dissever file uploads into two large groups, depending on whether the files uploaded by users are intended for public use, or they are private to each user. The avatar images discussed several times in this article are clearly in the commencement group, as these avatars are intended to be publicly shared with other users. On the other side, an application that performs editing operations on uploaded images would probably exist in the second group, considering y'all'd desire each user to only have access to their own images.

Consuming public uploads

When images are of a public nature, the easiest way to make the images bachelor for use by the awarding is to put the upload directory within the awarding's static folder. For example, an avatars subdirectory tin be created inside static, and so avatar images tin be saved at that place using the user id as proper name.

Referencing these uploads stored in a subdirectory of the static folder is done in the same mode as regular static files of the application, using the url_for() function. I previously suggested using the user id as a filename, when saving an uploaded avatar image. This was the way the images were saved:

          uploaded_file.salve(bone.path.bring together('static/avatars', current_user.get_id()))

With this implementation, given a user_id, the URL for the user's avatar can be generated as follows:

          url_for('static', filename='avatars/' + str(user_id))

Alternatively, the uploads can be saved to a directory exterior of the static folder, and and so a new route tin be added to serve them. In the example app.py awarding file uploads are saved to the location ready in the UPLOAD_PATH configuration variable. To serve these files from that location, we can implement the following route:

          from flask import send_from_directory  @app.road('/uploads/<filename>') def upload(filename):     return send_from_directory(app.config['UPLOAD_PATH'], filename)

One advantage that this solution has over storing uploads inside the static folder is that here you can implement boosted restrictions before these files are returned, either directly with Python logic inside the torso of the function, or with decorators. For example, if you desire to but provide access to the uploads to logged in users, you tin add Flask-Login's @login_required decorator to this road, or any other hallmark or function checking mechanism that you lot utilize for your normal routes.

Let's use this implementation idea to show uploaded files in our example application. Here is a new complete version of app.py:

          import imghdr import os from flask import Flask, render_template, request, redirect, url_for, abort, \     send_from_directory from werkzeug.utils import secure_filename  app = Flask(__name__) app.config['MAX_CONTENT_LENGTH'] = 1024 * 1024 app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] app.config['UPLOAD_PATH'] = 'uploads'  def validate_image(stream):     header = stream.read(512)  # 512 bytes should be plenty for a header check     stream.seek(0)  # reset stream pointer     format = imghdr.what(None, header)     if not format:         return None     return '.' + (format if format != 'jpeg' else 'jpg')  @app.route('/') def index():     files = os.listdir(app.config['UPLOAD_PATH'])     render render_template('alphabetize.html', files=files)  @app.road('/', methods=['POST']) def upload_files():     uploaded_file = asking.files['file']     filename = secure_filename(uploaded_file.filename)     if filename != '':         file_ext = os.path.splitext(filename)[ane]         if file_ext not in app.config['UPLOAD_EXTENSIONS'] or \                 file_ext != validate_image(uploaded_file.stream):             abort(400)         uploaded_file.save(os.path.join(app.config['UPLOAD_PATH'], filename))     return redirect(url_for('index'))  @app.route('/uploads/<filename>') def upload(filename):     return send_from_directory(app.config['UPLOAD_PATH'], filename)

In addition to the new upload() function, the alphabetize() view office gets the list of files in the upload location using os.listdir() and sends it downwardly to the template for rendering. The index.html template updated to show uploads is shown below:

          <!doctype html> <html>   <head>     <championship>File Upload</title>   </head>   <body>     <h1>File Upload</h1>     <class method="POST" action="" enctype="multipart/class-data">       <p><input type="file" proper noun="file"></p>       <p><input type="submit" value="Submit"></p>     </grade>     <60 minutes>     {% for file in files %}       <img src="{{ url_for('upload', filename=file) }}" mode="width: 64px">     {% endfor %}   </body> </html>

With these changes, every time yous upload an image, a thumbnail is added at the bottom of the page:

Consuming private uploads

When users upload private files to the application, additional checks need to exist in identify to prevent sharing files from i user with unauthorized parties. The solution for these cases require variations of the upload() view function shown above, with additional access checks.

A common requirement is to only share uploaded files with their possessor. A convenient way to store uploads when this requirement is present is to use a dissever directory for each user. For example, uploads for a given user tin exist saved to the uploads/<user_id> directory, and and then the uploads() function tin can be modified to merely serve uploads from the user'southward own upload directory, making information technology impossible for one user to come across files from another. Below you tin can run into a possible implementation of this technique, once once more bold Flask-Login is used:

          @app.route('/uploads/<filename>') @login_required def upload(filename):     return send_from_directory(os.path.join(         app.config['UPLOAD_PATH'], current_user.get_id()), filename)

Showing upload progress

Upward until now we have relied on the native file upload widget provided by the web browser to initiate our file uploads. I'm sure we can all agree that this widget is not very appealing. Not just that, but the lack of an upload progress display makes information technology unusable for uploads of large files, as the user receives no feedback during the entire upload process. While the telescopic of this commodity is to comprehend the server side, I idea information technology would be useful to give you a few ideas on how to implement a modern JavaScript-based file upload widget that displays upload progress.

The good news is that on the server there aren't any big changes needed, the upload mechanism works in the same style regardless of what method you apply in the browser to initiate the upload. To show you an example implementation I'm going to supersede the HTML class in index.html with 1 that is uniform with dropzone.js, a popular file upload client.

Here is a new version of templates/alphabetize.html that loads the dropzone CSS and JavaScript files from a CDN, and implements an upload class according to the dropzone documentation:

          <html>   <head>     <title>File Upload</title>     <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/dropzone/v.7.1/min/dropzone.min.css">   </caput>   <body>     <h1>File Upload</h1>     <form action="{{ url_for('upload_files') }}" class="dropzone">     </course>     <script src="https://cdnjs.cloudflare.com/ajax/libs/dropzone/5.7.1/min/dropzone.min.js"></script>   </body> </html>

The 1 interesting thing that I've constitute when implementing dropzone is that it requires the action attribute in the <form> element to exist fix, even though normal forms accept an empty activeness to indicate that the submission goes to the same URL.

Outset the server with this new version of the template, and this is what yous'll become:

That's basically it! You lot can now drib files and they'll be uploaded to the server with a progress bar and a final indication of success or failure.

If the file upload fails, either due to the file being too large or invalid, dropzone wants to display an fault message. Because our server is currently returning the standard Flask error pages for the 413 and 400 errors, you will see some HTML gibberish in the error popup. To correct this we tin can update the server to render its error responses every bit text.

The 413 mistake for the file besides big condition is generated by Flask when the asking payload is bigger than the size set in the configuration. To override the default fault page nosotros have to use the app.errorhandler decorator:

          @app.errorhandler(413) def too_large(e):     render "File is as well big", 413

The second error status is generated by the application when any of the validation checks fails. In this example the error was generated with a abort(400) telephone call. Instead of that the response tin can be generated directly:

                      if file_ext not in app.config['UPLOAD_EXTENSIONS'] or \                 file_ext != validate_image(uploaded_file.stream):             return "Invalid image", 400

The terminal modify that I'grand going to brand isn't actually necessary, but it saves a bit of bandwidth. For a successful upload the server returned a redirect() back to the main route. This caused the upload form to exist displayed again, and also to refresh the listing of upload thumbnails at the bottom of the folio. None of that is necessary now because the uploads are done as background requests past dropzone, so we can eliminate that redirect and switch to an empty response with a lawmaking 204.

Here is the complete and updated version of app.py designed to piece of work with dropzone.js:

          import imghdr import os from flask import Flask, render_template, asking, redirect, url_for, arrest, \     send_from_directory from werkzeug.utils import secure_filename  app = Flask(__name__) app.config['MAX_CONTENT_LENGTH'] = two * 1024 * 1024 app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] app.config['UPLOAD_PATH'] = 'uploads'  def validate_image(stream):     header = stream.read(512)     stream.seek(0)     format = imghdr.what(None, header)     if non format:         return None     return '.' + (format if format != 'jpeg' else 'jpg')  @app.errorhandler(413) def too_large(e):     return "File is too large", 413  @app.route('/') def index():     files = os.listdir(app.config['UPLOAD_PATH'])     render render_template('index.html', files=files)  @app.route('/', methods=['POST']) def upload_files():     uploaded_file = request.files['file']     filename = secure_filename(uploaded_file.filename)     if filename != '':         file_ext = os.path.splitext(filename)[one]         if file_ext non in app.config['UPLOAD_EXTENSIONS'] or \                 file_ext != validate_image(uploaded_file.stream):             return "Invalid image", 400         uploaded_file.salvage(os.path.bring together(app.config['UPLOAD_PATH'], filename))     render '', 204  @app.route('/uploads/<filename>') def upload(filename):     render send_from_directory(app.config['UPLOAD_PATH'], filename)

Restart the application with this update and now errors volition have a proper bulletin:

The dropzone.js library is very flexible and has many options for customization, then I encourage you lot to visit their documentation to learn how to adapt it to your needs. Yous can also look for other JavaScript file upload libraries, as they all follow the HTTP standard, which means that your Flask server is going to piece of work well with all of them.

Conclusion

This was a long overdue topic for me, I tin can't believe I have never written anything on file uploads! I'd love you hear what you call back about this topic, and if y'all think there are aspects of this characteristic that I haven't covered in this commodity. Feel gratis to let me know beneath in the comments!