mon petit blogue sans importance...

django ajax comments with reCAPTCHA

It turns out that the default django comments mechanisms for fighting spam are not very effective at all, contrary to my expectations. So i had to bring reCAPTCHA back. But then i figured i might as well do some kind of ajax comment form if i'm going to rewrite my commenting. A few problems cropped up, and it's still ugly, but it works now, and i haven't had a single spam comment since, whereas i consistently had a couple a day before.

So i wanted to bring back reCAPTCHA for my comments, as i find it's the most effective spam-fighting method and it helps other people, too. But i don't want the reCAPTCHA to be loaded for every view of the blog post page - that's too wasteful, even though at my level of traffic it's totally insignificant. My previous solution had been to have a separate page that would preview your comment, where the reCAPTCHA would be shown, but that's awkward: it required 3 page views to enter a comment! (blog post page, preview, "your comment has been submitted", "see comment in context" (back to blog post page)).

So the solution was a very simplistic one. The base template for a blog post page contains an empty <div> for the comment form. I then created a partial view. A partial view is simply a template+ view, just like any other, except that instead of returning a full html document, it returns a snippet, which has no <html>, <head> or <body> tags. Javascript functions in other pages will then request this "partial" and insert it into the main document somewhere.

Then there's a button or link (a link in this case, "show comment form") which triggers this javascript. The js will then fetch the partial and insert it into my formerly empty comment form div tag. The partial returns a full comment form, including the reCAPTCHA.... or at least, it did originally. Back to this later.

My problem with the previous incarnation of a reCAPTCHA comment form was that i had just copied the view function from the django comment contrib application and modified it, which is just stupid. Now, i created an actual django form, which inherits from the upstream comment form. It just adds the proper validation for the reCAPTCHA:

class ReCaptchaCommentForm(CommentForm):
    def __init__(self, remote_ip, *args, **kwargs):
        self.remote_ip = remote_ip
        self.recaptcha_error = ''
        super(ReCaptchaCommentForm, self).__init__(*args, **kwargs)
    
    def recaptcha_valid(self):
        conn = httplib.HTTPConnection("api-verify.recaptcha.net")
        if 'recaptcha_challenge_field' not in self.data or \
           'recaptcha_response_field' not in self.data:
            # spammer!
            return False
        params = urllib.urlencode({'privatekey':settings.RECAPTCHA_PRIVATE,
                                   'remoteip':self.remote_ip,
                                   'challenge':self.data['recaptcha_challenge_field'],
                                   'response':self.data['recaptcha_response_field'].encode('utf-8')
                                  })
        headers = {"Content-type": "application/x-www-form-urlencoded",
                   "Accept": "text/plain"}
        conn = httplib.HTTPConnection("api-verify.recaptcha.net:80")
        conn.request("POST", "/verify", params, headers)
        recatpcha_resp = conn.getresponse()
        if recatpcha_resp.status != httplib.OK:
            self.recaptcha_error = 'recaptcha-not-reachable'
        else:
            try:
                a, self.recaptcha_error = recatpcha_resp.read().split('\n')
            except httplib.HTTPException:
                self.recaptcha_error = 'recaptcha-not-reachable'
            if self.recaptcha_error not in ("", "incorrect-captcha-sol"):
                from django.core.mail import mail_admins
                mail_admins("Bad response status from recaptcha server", str(self.data) 
                            + "\n\n" + str(recatpcha_resp))
                print "recatpcha problem " + self.recaptcha_error
        return self.recaptcha_error == 'success'

I then added a view which does duplicate what the contrib comment view does, but it was the simplest way - it's a simple view to begin with. It does the form checks, returns the comment form but with errors and fields already filled in if there were errors with the validation, and otherwise will return a rendered comment, using another partial which renders a single comment.

def comment_form(request, queryset, extra_context={}, year=None, month=None, day=None,
                 slug='', template='blog/comment_form.html'):
    try:
        # python 2.4 doesn't do datetime.strptime
        t = time.strptime(year+month+day, '%Y%m%d')
        date = dt.date(*t[:3])
    except ValueError:
        raise Http404
    lookup_kwargs = {'pub_date__range': (dt.datetime.combine(date, dt.time.min),
                                         dt.datetime.combine(date, dt.time.max))}
    lookup_kwargs['slug'] = slug
    entry = get_object_or_404(Entry, **lookup_kwargs)
    
    context = {}
    if request.method == 'POST':
        form = ReCaptchaCommentForm(request.META['REMOTE_ADDR'], entry, request.POST)
        preview = 'preview' in request.POST
        if form.is_valid():
            c = form.get_comment_object()
            if preview:
                context['comment'] = c
            elif request.user.is_authenticated() or form.recaptcha_valid():
                c.save()
                context['comment'] = c
                print "posted comment"
                # instead of returning a new form, we return the rendered comment
                template = 'blog/comment_partial.html'
            else:
                # not in preview mode, but captcha didn't validate
                pass
    else:
        # return a brand new comment form
        form = ReCaptchaCommentForm(request.META['REMOTE_ADDR'], entry)
        if request.user.is_authenticated():
            form.initial['name'] = request.user.username
            form.initial['email'] = request.user.email
    
    context.update({
        'recaptcha_public': settings.RECAPTCHA_PUBLIC,
        'recaptcha_error': _RECAPTCHA_ERRORS.get(form.recaptcha_error, _("unknown error")),
        'comment_form_target': request.path,
        'form': form
    })
    return render_to_response(template, context)

The rendered comment template is also used in the main blog post template to actually show all the posted comments when a visitor looks at the post - the same template is just included once for each comment found for this post, in a loop. Thus, the same "rendered comment" template is re-used.

This is far from the most effective way to do things. A much better way is that the "post comment" javascript POST call would not return complete html fragments, but just information about the comment submission: were there errors? if so, which? and if not, what does the rendered comment look like? Plus, when there are errors, you'll notice my layout wiggles, because it doesn't account for the errors when it's first rendered. But still, this is miles ahead of requiring a page reload for previewing the comment, then another each time there's an error in the entered data, and then another for posting it, and then another to see it in context.

Now, the biggest problem i had (most of this takes next to no time) was that initially, my comment form partial used the "Challenge and non-JavaScript API" (scroll down in the reCAPTCHA API doc page to see it), which is what i had used before. But whenever my js would request the partial and then insert it in the empty div, the page would reload to an empty page. Very frustrating. It turns out it's due to something the reCAPTCHA javascript does.

You have to use the "reCAPTCHA AJAX API" for this to work properly, which basically comes down to loading the reCAPTCHA js library when your main page loads, and when you need a reCAPTCHA to appear, you call a helper function - Recaptcha.create - and tell it in what div you want the reCAPTCHA challenge to appear. In my case, the comment form partial template has an empty div for this. When the template is requested by the client js (i.e., when the user clicked "show comment form"), it's rendered and sent back to the client browser without a reCAPTCHA challenge, just the html comment form. But then there's a callback in the js function that's called to fetch the form - update_comment_form -, which will ask the reCAPTCHA library to generate the challenge and insert it right after the comment form was inserted in the document. That same function also hides the comment form div first, so that this rendering doesn't cause a bunch of weird effects in the browser. When all is done, the form div is unhidden, and the whole form shows up.

In practice, the whole thing needs a lot of love, cause it's just not that pretty right now. But it works quite well.

To do the javascript part, i used jQuery: it just makes things a lot simpler.

    $(document).ready(function(){
        $("#get_comment_form_button").html("{% trans "show comment form" %}");
        $("#comment_form_content").hide();
        var scroll_to_form = function() {
            var offset = $("#comment_form").offset().top;
            $('html,body').animate({scrollTop: offset}, 1000);
        };
        var post_comment_form = function(preview) {
            $("#preview_comment_button").attr("disabled","true");
            $("#post_comment_button").attr("disabled","true");
            var f = $("#comment_form");
            var f_serial = f.serialize();
            if (preview=="true") {
                f_serial = f_serial+"&preview=true";
                $("#preview_comment_button").val("{% trans "loading..." %}")
            } else {
                $("#post_comment_button").val("{% trans "loading..." %}")
            }
            var action = f.attr("action");
            $.post(action, f_serial, update_comment_form);
        }
        var update_comment_form = function(resp, status) {
            $("#get_comment_form_content").hide();
            $("#comment_form_content").html(resp);
            Recaptcha.create("{{ recaptcha_public }}", "recaptcha", {
                theme: "red",
                tabindex: 0,
                lang: "{{ LANGUAGE_CODE }}",
                error: $("#recaptcha_error").html(),
                callback: scroll_to_form
            });
            $("#comment_form_content").show();
            //$("#comment_form").submit(function(e){
            $("#preview_comment_button").click(function(e){
                e.preventDefault();
                post_comment_form("true");
                return false;
            });
            $("#post_comment_button").click(function(e){
                e.preventDefault();
                post_comment_form("false");
                return false;
            });
            return false;
        };
        $("#get_comment_form_button").click(function(e) {
            e.preventDefault();
            $("#get_comment_form_content").hide();
            $("#get_comment_form_content").html("{% trans "loading..." %}");
            $("#get_comment_form_content").show();
            $.get("comments/form/", {}, update_comment_form);
            $("#get_comment_form_button_wrap").empty();
            return false;
        });
    });

I'm sure there's a gazillion things wrong / ugly / suboptimal with this, and i know there's some superfluous content wiping steps, and i know it could be made prettier, but it does the job. Do suggest improvements in the comments if you feel like it.

Thanks to Ben for the little help when i had some problems with reCAPTCHA screwing up my partial view!

by wiswaud on 24 March 2009
Tags: django, english, geeky, metablogging, python, web

mon petit blogue sans importance...

django ajax comments with reCAPTCHA

Comments

Twitter Updates