Using django-importcsvadmin to Streamline Excel Imports

 

If you've spent much time at all in web development, you've probably at some point needed to import a large chunk of data from Excel. Django makes this easy: export the .xls file to a .csv file, parse the csv data using Python's standard library csv module, and then create a bunch of models. This works fine if you only need to import the data once, or if your website's admin is a programmer, but this often isn't the situation.

Excel is the de-facto data storage tool for businesses. While the web development purist in me might argue for switching from Excel to a relational database, the truth is this is often neither practical nor desirable. As a result, site admins often need a way to update their websites using their Excel sheets. django-importcsvadmin is a lightweight, no-nonsense tool for adding csv import to your Django admin to make this easy.

Here's a simple example: Let's suppose we have an Excel spreadsheet for products withname, price and description columns, and your Product model has corresponding columns. Just add importcsvadmin to your INSTALLED_APPS, and add the following code to your admin.py:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

from django.contrib import admin
from django import forms

from importcsvadmin.admin import ImportCSVModelAdmin

from my_app.models import Product


class ProductAdminImporter(forms.ModelForm):
    class Meta:
        model = Product
        fields = ('name', 'price', 'description')


class ProductAdmin(ImportCSVModelAdmin):
    importer_class = ProductAdminImporter


admin.site.register(Product, ProductAdmin)

And that's it! In the Admin for your Product model, now you'll have a working "Import CSV" button, and your non-programmer admins now have an easy and convenient way to keep their website's database synced up with their familiar spreadsheet. The spreadsheet will be processed row-by-row (in an atomic transaction of course) and each row will be validated by the ProductAdminImporter form before import.

The key idea behind django-importcsvadmin is that Django ModelForm are the correct tool for validating csv data. If you're used to thinking of forms as primarily things you display on a web page, this might seem a little strange, but really Django's forms are primarily a tool for validating and recording data. Seen in that light they're a nearly ideal tool for importing csv data. By using a ModelForm we leverage Django's powerful form validation system, including sensible default validation rules and flexible customization. Forms are Django's primary mechanism for adding data to your database, and I would go so far as to say that any time you're not using a form to import your data, you should give your approach some serious thought.

So instead of writing complex and redundant validation rules for our csv data, we can rely on our Model definition to do most of the work for us. In simple cases like the one above, the system will work with basically no customization. Since ImportCSVModelAdmin is just aModelAdmin with a few features added we can configure it just like we would any otherModelAdmin. And, since our importer is just a ModelForm we can add custom validation easily. If the data in the spreadsheet needs some preprocessing, we can do it the same way we would for any user submitted data - using custom fields. For more complex cases almost anything can be subclassed. Since the app is generic and lightweight for more complex projects you can think of it as an easy scaffold for building a csv importer; subclass the admin, change whatever methods or attributes you need to for your purposes, and start importing.

Check it out!