Django tutorial: as-you-type search with Ajax

This is a walkthrough tutorial on how to implement what’s defined as “incremental search” in a Django app. We want results to refresh (with a tiny delay) as the user types their search term. We’ll also give a visual indication that the search is running by animating the search icon.

Here’s a demo of the final functionality:

The source code for this tutorial is available on Github.

Our app

Don’t worry, not another blog or to-do app. This time it’s a music website that displays music albums and artists. The structure is taken from an actual web-app I’ve built but is simplified for this post’s purposes. Here’s a folder only view:

django-ajax-search/
├── core
│   └── migrations
├── django-ajax-search
├── static
│   └── django-ajax-search
│       └── javascript
└── templates

Our project’s root directory is called django-ajax-search, and we’ve created an app called core where we’ll write most of our Django-related code. Make sure it’s included in INSTALLED_APPS in your settings.py file.

Here’s the models.py file:

# core/models.py
class MusicRelease(models.Model):
    title = models.CharField(max_length=560)
    release_date = models.DateField(blank=True, null=True)  # some releases don't have release-dates

    def __str__(self):
        return self.title

    @property
    def is_released(self):
        return self.release_date < timezone.now().date()


class Artist(models.Model):
    music_releases = models.ManyToManyField(MusicRelease, blank=True)
    name = models.CharField(max_length=560)

    def __str__(self):
        return f"{self.name} (release count: {self.music_releases.count()})"

Nothing fancy: each artist can have many music releases, and each release can have many artists. I’ve also added some helpful string representations to the Artist model.

High-level overview

We’re going to let users search for artists in the database by name. Instead of a form with a submit button, we’re going to refresh the results as the user types their query.

There are several moving pieces in this article so here’s an outline of what we’re going to do:

1. Briefly go over how HTTP GET parameters are handled in Django views and make our view capture the user’s query.
2. Make the Django view handle Ajax requests and respond to them properly with a JSON response containing the new results.
3. Use JavaScript and jQuery to send an Ajax request to our view once the user starts typing in the HTML search box. This request will include the term so the server can return relevant results.
4. Once our view returns the JSON response, our JS code will use it to change the information presented to the user without a page-refresh.

Some of the concepts above will be discussed verbatim and others will be covered briefly.

Dependencies and additional setup

Make sure jQuery (CDN link) is included inside the head tag of the base.html template. While they’re not strictly required, I’ll also be using Bootstrap 4 as a CSS framework and Font Awesome for the search icon, which we’ll make blink when a search is being taken care of by the server.

Another thing to verify is that the JS file is included in base.html:

{# base.html #}
{% block footer %}
  <script type="text/javascript" src="{% static "javascript/main.js" %}"></script>
{% endblock %} 

Again, you don’t have to follow the structure religiously but if you’re ever confused on where things belong, check out the Github repository.

Artists in our database

Let’s also create artists to work with. The demo app is going to have three:

Chet Faker (release count: 1)
Queen (release count: 1)
Parker Sween (release count: 0)

I don’t know who Parker Sween is.

The artists view

Here’s the views.py file:

# core/views.py
def artists_view(request):
    ctx = {}
    url_parameter = request.GET.get("q")

    if url_parameter:
        artists = Artist.objects.filter(name__icontains=url_parameter)
    else:
        artists = Artist.objects.all()

    ctx["artists"] = artists

    return render(request, "artists.html", context=ctx)

This view is referenced like this in our urls.py:

# urls.py
from django.urls import path
from core import views as core_views

urlpatterns = [
    # ...
    path("artists/", core_views.artists_view, name="artists"),
]

So the path ourapp.com/artists/ is going to hit this view. Let’s pick it further apart.

Capturing HTTP GET parameters

The first thing to make sure of is that the view captures the GET parameter we’re going to send. Here’s the relevant line:

def artists_view(request):
    # ...
    url_parameter = request.GET.get("q")

So, one way to pass information between clients and servers is using HTTP GET parameters:

https://www.somewebsite.com/some-page?name=josh

When a URL like the above is requested, the server will receive the GET parameter name alongside its value josh. It’s up to the server to decide what to do with this parameter, if at all.

In Django views, these URL GET parameters are made available in a special kind of dictionary — a QueryDict called GET. This QueryDict lives in the request object, the one every Django view accepts as its first argument. Going back to the line above:

def artists_view(request):
    # ...
    url_parameter = request.GET.get("q")

This means that our view will capture a GET parameter q. If it isn’t passed at all, url_parameter will be None. The first GET is the dictionary itself, and the second get() is just the method used to retrieve a key’s value from a dictionary.

Some examples of URLs requested and how they would map:

URL requested: https://ourapp.com/artists?q=Queen
url_parameter value: "Queen"

URL requested: https://ourapp.com/artists?q=Samba
url_parameter value: "Samba"

URL requested: https://ourapp.com/artists?q=Chet Faker (decoded)
url_parameter value: "Chet Faker"

URL requested: https://ourapp.com/artists/?q=Chet%20Faker (encoded)
url_parameter value: "Chet Faker"

Case insensitive filtering

Another portion to go over in artists_view:

# core/views.py
def artists_view(request):
# ...
if url_parameter:
    artists = Artist.objects.filter(name__icontains=url_parameter)
else:
    artists = Artist.objects.all()

If url_parameter’s value isn’t None, it means that some string was passed after ?q= and we want to filter for Artist objects containing this string. Using icontains means the search will also be case-insensitive. For example: if url_parameter is KER, our view will return a QuerySet containing two of our artists: Chet Faker and Parker Sween. Queen won’t be there.

Template files

Our view renders the template file artists.html:

{# artists.html #}
{% extends "base.html" %}

{% block content %} 
<h3>Artists</h3>

<div class="row">

  {# icon and search-box #}
  <div class="col-6 align-left">
    <i id="search-icon" class="fas fa-search"></i>
    <input id="user-input" placeholder="Search">
  </div>

  {# artist-list section #}
  <div id="replaceable-content" class="col-6">
    {% include 'artists-results-partial.html' %}
  </div>

</div>
{% endblock %}

The first thing to note is that this template includes another template, artists-results-partial.html:

{# artists-results-partial.html #}
{% if artists %}
  <ul>
  {% for artist in artists %}
    <li>{{ artist.name }}</li>
  {% endfor %}
  </ul>
{% else %}
  <p>No artists found.</p>
{% endif %}

Including the artist-list in a separate template partial doesn’t only yield better readability; more importantly, it will allow us to more easily refresh this part (and this part only) of the page using JavaScript & jQuery. Also, take note of the HTML id attributes we assign to each of the search icon, the input field, and the div holding our artist list. We will use these values later when we target these elements for manipulation with jQuery.

Making the view respond to Ajax requests

Before we get to the JS code, there’s one last addition we need to make in artists_view so it responds to Ajax requests:

from django.template.loader import render_to_string
from django.http import JsonResponse

def artists_view(request):
    # ...earlier code
    is_ajax_request = request.headers.get("x-requested-with") == "XMLHttpRequest" and does_req_accept_json
    
    if is_ajax_request:
        html = render_to_string(
            template_name="artists-results-partial.html", 
            context={"artists": artists}
        )

        data_dict = {"html_from_view": html}

        return JsonResponse(data=data_dict, safe=False)

    return render(request, "artists.html", context=ctx)

We first check if the request was made via an Ajax call. In this case, we want to return the browser a JSONResponse. But what are we returning, exactly?

We’re passing JSONResponse a dictionary we’ve constructed, called data_dict. It has a single key html_from_view. This key’s value is going to be the variable html.

html is our template artists-results-partial.html rendered as a string. It literally is the HTML output of our artist-list. We provide Django’s render_to_string() a template to use and a context dictionary, and it returns to us that template as a string given the context it was fed. If it’s not clear yet, here’s an example:

In the view, If the variable artists is this QuerySet:

<QuerySet [<Artist: Chet Faker (release count: 1)>]>

Then these lines:

html = render_to_string(
            template_name="artists-results-partial.html", 
            context={"artists": artists}
        )
print(html)

Will print the following:

<ul>
  <li>Chet Faker</li>
</ul>

You can see where this is going by now: using JS and jQuery, we can pass whatever the user is typing in the input box to our view as a GET parameter, filter by that string, and then return a JSON response with the new HTML to the browser where it will replace the old HTML.

Implementing Ajax search

We’re going to send an Ajax request to the server. Once we get a JSON response back, we’ll use jQuery to manipulate the relevant HTML elements. I can’t possibly go in-depth on each piece of functionality here as that’s beyond the scope of this post but I’ll try to at least explain the bigger picture.

Ajax (AJAX) stands for Asynchronous Javascript and XML. The key word here is asynchronous: it allows to send and receive data between clients (browsers) and servers without the need to reload the entire page.

jQuery is one of the most popular JavaScript libraries, to the point where some would confuse it as a language on its own. Its mission statement is “to allow developers to do more while writing less code”.

Here’s the full JavaScript code:

const user_input = $("#user-input")
const search_icon = $('#search-icon')
const artists_div = $('#replaceable-content')
const endpoint = '/artists/'
const delay_by_in_ms = 700
let scheduled_function = false

let ajax_call = function (endpoint, request_parameters) {
    $.getJSON(endpoint, request_parameters)
        .done(response => {
            // fade out the artists_div, then:
            artists_div.fadeTo('slow', 0).promise().then(() => {
                // replace the HTML contents
                artists_div.html(response['html_from_view'])
                // fade-in the div with new contents
                artists_div.fadeTo('slow', 1)
                // stop animating search icon
                search_icon.removeClass('blink')
            })
        })
}


user_input.on('keyup', function () {

    const request_parameters = {
        q: $(this).val() // value of user_input: the HTML element with ID user-input
    }

    // start animating the search icon with the CSS class
    search_icon.addClass('blink')

    // if scheduled_function is NOT false, cancel the execution of the function
    if (scheduled_function) {
        clearTimeout(scheduled_function)
    }

    // setTimeout returns the ID of the function to be executed
    scheduled_function = setTimeout(ajax_call, delay_by_in_ms, endpoint, request_parameters)
})

Let’s look at the first few lines:

const user_input = $("#user-input")
const search_icon = $('#search-icon')
const artists_div = $('#replaceable-content')

Remember how we gave some of the HTML elements in artists.html an ID attribute? Here, we’re using a jQuery selector to save those elements as variables so we can more easily refer to them later in the code. All jQuery selectors start with a dollar sign with the selected arguments enclosed in parenthesis.

We’re then initializing some additional variables:

const endpoint = '/artists/'
const delay_by_in_ms = 700
let scheduled_function = false

The first one is the relative path to the endpoint we’re going to make our Ajax request to. Note that this has to be a path where a Django URL is defined and we should always use the URL path because JavaScript knows nothing about Django’s named URLs or views.

scheduled_function and delay_by_in_ms are explained later.

After the variables, we define the function ajax_call() which we invoke towards the end of the code:

let ajax_call = function (endpoint, request_parameters) {
    $.getJSON(endpoint, request_parameters)
        .done(response => {
            // fade out the artists_div, then:
            artists_div.fadeTo('slow', 0).promise().then(() => {
                // replace the HTML contents
                artists_div.html(response['html_from_view'])
                // fade-in the div with new contents
                artists_div.fadeTo('slow', 1)
                // stop animating search icon
                search_icon.removeClass('blink')
            })
        })
}

This one takes two arguments, endpoint and request_parameters. It then uses jQuery’s getJSON() method to send an Ajax request to the endpoint alongside the parameters. When it’s done, it’s going to give us an object we call response. We then fade out artists_div, replace its contents with response['html_from_view'], and fade it back-in. If you’re confused about where html_from_view is coming from, go back to the view code responsible for handling Ajax requests.

Once the function is defined, we’re using jQuery’s on() to bind a function to each keyup event that happens on user_input:

user_input.on('keyup', function () {
  // our code
})

This means that each time a keyboard key is released (after being pressed) inside user_input, the function is run. Let’s inspect this function’s body:

const request_parameters = {
    q: $(this).val() // value of user_input: the HTML element with ID user-input
}

The first step is getting the value inside the input field. This is the string the user has typed so far. We save it inside an object request_parameters  where its key is q.

Next, we add the blink CSS class to our search icon:

// start animating the search icon with the CSS class
search_icon.addClass('blink')

This lets the user know we’re doing something with their request. The search icon will blink indefinitely as long as it has this class. That’s why we remove it at the end of ajax_call().

Here’s the CSS code defining blink:

@keyframes blinker {
    from {opacity: 1.0;}
    to {opacity: 0.0;}
  }
  
.blink {
    text-decoration: blink;
    animation-name: blinker;
    animation-duration: 0.6s;
    animation-iteration-count:infinite;
    animation-timing-function:ease-in-out;
    animation-direction: alternate;
  }

Making sure the server isn’t hammered

Now, to the setTimeout/clearTimeout part:

// if scheduled_function is NOT false, cancel the execution of the function
if (scheduled_function) {
    clearTimeout(scheduled_function)
}

// setTimeout returns the ID of the function to be executed
scheduled_function = setTimeout(ajax_call, delay_by_in_ms, endpoint, request_parameters)

setTimeout() is a built-in JavaScript function that delays a function execution by a predefined duration (specified in milliseconds). It returns an ID of the function that is scheduled for execution. Here’s its signature:

setTimeout(func, delay_in_ms, func_param1, func_param2, ...)

Compare this with the parameters we’re passing above and you can see that the code is scheduling the ajax_call() function to execute after 700 milliseconds.

But why introduce a delay?

Because if we actually hit the server every time the keyup event is registered, we’re going to flood it with too many requests in a short amount of time. Here’s an example of a naïve implementation that doesn’t use setTimeout() and clearTimeout(). I’ve added a logging message that prints to the console each time a request is made:

So yeah, we don’t want to hammer the server with every keystroke like that. That’s what we utilize setTimeout() for. But that’s only one part of the puzzle. With setTimeout(), all we’re doing is delaying the execution of each query by 700ms. What we really want to do is send a request only after the user has ceased typing for a bit. That’s where clearTimeout() comes in.

clearTimeout() is another built-in function. Given a function ID returned by setTimeout(), it cancels the execution of that function if it hasn’t already been executed.

Now let’s look at that piece of code again:

// if scheduled_function is NOT false, cancel the execution of the function
if (scheduled_function) {
    clearTimeout(scheduled_function)
}

// setTimeout returns the ID of the function to be executed
scheduled_function = setTimeout(ajax_call, delay_by_in_ms, endpoint, request_parameters)

The above code block simply ensures an Ajax call is sent to our server at most every 700 milliseconds.

Since we initialized scheduled_function to false, the first time the if statement is evaluated, it’s going to skip clearTimeout() and instantly schedule our Ajax call to execute after 700ms and that function’s ID in scehduled_function.

Now, if within that very short timespan (699 milliseconds) the user types another letter, and since the variable scheduled_function is now truthy, the if block will evaluate to true and clearTimeout() will cancel the function that was scheduled for execution. Instantly after that, another new Ajax call is scheduled, and the cycle continues…if 700 milliseconds did pass since the user had last typed anything, ajax_call() is executed normally.

If you’re still grappling with this concept, try to think of setTimeout() as scheduleFunction() and clearTimeout() as cancelScheduledFunction().

Summary

Nowadays using Django as a backend with a frontend framework like Vue.js or React is all the rage, but sometimes all that’s needed for interactivity in a “classic” Django app is some JavaScript and jQuery knowledge.

You can clone the Github repository and play around with the working code if you feel you need a better understanding of some of the concepts outlined.