[Django] #30514: Occasional CSRF cookie not set Django 2.1

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[Django] #30514: Occasional CSRF cookie not set Django 2.1

Django
#30514: Occasional CSRF cookie not set Django 2.1
----------------------------------------+------------------------
               Reporter:  Yusuf Musleh  |          Owner:  nobody
                   Type:  Bug           |         Status:  new
              Component:  CSRF          |        Version:  2.1
               Severity:  Normal        |       Keywords:
           Triage Stage:  Unreviewed    |      Has patch:  0
    Needs documentation:  0             |    Needs tests:  0
Patch needs improvement:  0             |  Easy pickings:  0
                  UI/UX:  0             |
----------------------------------------+------------------------
 Hi all,

 We recently noticed an increase in the number of 403 errors we were
 getting in our server, mainly the `CSRF cookie not set.` We added more
 logging and investigated it to see if they were legitimate errors
 (requests that should cause a 403 error), some are, however we discovered
 that a lot of them aren't. There were normal requests coming from normal
 users, we would get around 30 of these failed requests in a given day, it
 would happen randomly across random devices, browsers and urls and has
 been quite difficult to reproduce.

 We Copy/Pasted the source code of `middleware.csrf` and added it to our
 code base as a custom middleware to add more logging and get a better
 traceback in sentry when the error occurs, to get more insights on issue,
 we learned the following:

 - For some reason the token was not set. `csrf_token =
 request.META.get('CSRF_COOKIE’)` returns `None`,
 - We know that if a user got the error, if they simply refresh the page
 things would work perfectly fine, this means that setting the token works,
 but sometimes it does not
 - We know that it is not a problem with our frontend since we also got
 this error in the django admin including non-login requests
 - We tried clearing all the expired sessions, and set `CSRF_USE_SESSIONS =
 True`, in hopes of it improving the situation, but nothing much has
 changed.

 With all that being said, we narrowed down the problem to the
 `_get_token(self, request)` method, since we set `CSRF_USE_SESSIONS =
 True` it tries to fetch the token from the session using
 `request.session.get(CSRF_SESSION_KEY)`, but that would return `None`. A
 very interesting thing we noticed is that the CSRF token **is** present in
 the actual request's header under `HTTP_X_CSRFTOKEN`. So we modified the
 method to fetch the CSRF token from the header as a fallback to see if it
 would fix the issue:

 {{{
 def _get_token(self, request):
     if settings.CSRF_USE_SESSIONS:
         try:
             token_from_session = request.session.get(CSRF_SESSION_KEY)
         except AttributeError:
             raise ImproperlyConfigured(
                 'CSRF_USE_SESSIONS is enabled, but request.session is not
 '
                 'set. SessionMiddleware must appear before
 CsrfViewMiddleware '
                 'in MIDDLEWARE%s.' % ('_CLASSES' if settings.MIDDLEWARE is
 None else '')
             )
     else:
         try:
             cookie_token = request.COOKIES[settings.CSRF_COOKIE_NAME]
         except KeyError:
             return None

         csrf_token = _sanitize_token(cookie_token)
         if csrf_token != cookie_token:
             # Cookie token needed to be replaced;
             # the cookie needs to be reset.
             request.csrf_cookie_needs_reset = True
         return csrf_token

     # Fallback code we added to fetch the token from the header
     if token_from_session is None:
         token_from_session = request.META.get(settings.CSRF_HEADER_NAME)

     return token_from_session
 }}}

 Fortunately after this change was made, we haven't gotten any `CSRF cookie
 not set.` errors since then because the `_get_token` method now returns a
 token, and the `process_request` method goes smoothly:

 {{{
 def process_request(self, request):
     csrf_token = self._get_token(request)
     if csrf_token is not None:
         # Use same token next time.
         request.META['CSRF_COOKIE'] = csrf_token
 }}}

 the `request.META.get('CSRF_COOKIE’)` no longer returns `None`.

 We also tested sending bogus requests, ones with no csrf tokens set and
 ones with incorrect ones just to make sure that the CSRF functionality
 still works properly, and we get the `CSRF cookie not set.` and `CSRF
 token missing or incorrect.` respectively as expected.

 The question still remains, we're not sure why this bug happens. The
 problem is since django would consider these requests as bogus and handles
 them accordingly, we were not aware of the issue for a while. Until it
 randomly happened to a few people on our team, where they would open a
 fresh page on our website and click on a button that would make a request,
 and the error would occur, thats when we began the investigation.

 Could it be possible that due of the nature of this bug, it is occurring
 to others as well, but they are just not aware of it? I tried to include
 as much information as possible, we'd appreciate if someone can shed some
 light on matter.

 Thanks.

--
Ticket URL: <https://code.djangoproject.com/ticket/30514>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/055.8be9d1aac94aaf23106b6de08c584c2b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #30514: Occasional CSRF cookie not set Django 2.1

Django
#30514: Occasional CSRF cookie not set Django 2.1
------------------------------+--------------------------------------
     Reporter:  Yusuf Musleh  |                    Owner:  nobody
         Type:  Bug           |                   Status:  closed
    Component:  CSRF          |                  Version:  2.1
     Severity:  Normal        |               Resolution:  needsinfo
     Keywords:                |             Triage Stage:  Unreviewed
    Has patch:  0             |      Needs documentation:  0
  Needs tests:  0             |  Patch needs improvement:  0
Easy pickings:  0             |                    UI/UX:  0
------------------------------+--------------------------------------
Changes (by Carlton Gibson):

 * status:  new => closed
 * resolution:   => needsinfo


Comment:

 Hi Yusuf,

 Thanks for the report. Very well explained, and interesting.

 To be honest, there's not much I can say immediately. There's two possible
 scenarios:

 1. There's a correct HTTP request, which Django is mishandling.
 2. Django's behaving, but the HTTP request is missing something.

 If you could put a test case together showing 1 then we'd having something
 we could directly address.

 Re 2, the only thing that immediately comes to mind is that we've seen
 Safari behaving ''strangely'' due to it's newer "Intelligent Tracking
 Protection". More or less, when using (multiple?) internal redirects,
 Safari will simply not send the cookies for the request, which would
 explain how your session lookup would be failing.

 There's django-developers thread on this topic here:
 https://groups.google.com/d/topic/django-developers/RyDdt1TcH0c/discussion

 Since you have the Sentry data available, can you see if perhaps Safari is
 the common theme here?
 If it's not, what else do the requests have in common?

 Maybe someone on [https://groups.google.com/forum/#!forum/django-users
 django-users] will have seen similar and can help to pin it down?

 As it stands I'm going to close this as `needsinfo`. I'm really happy to
 re-open if can show Django is somehow at fault.

 (If it is Safari that's at issue, it' worth continuing the django-
 developers discussion, because we should/could at least try to codify
 exactly what's happening and how to handle it...)

--
Ticket URL: <https://code.djangoproject.com/ticket/30514#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.72a77e8db113f15beeb76c93d9629271%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #30514: Occasional CSRF cookie not set Django 2.1

Django
In reply to this post by Django
#30514: Occasional CSRF cookie not set Django 2.1
------------------------------+--------------------------------------
     Reporter:  Yusuf Musleh  |                    Owner:  nobody
         Type:  Bug           |                   Status:  closed
    Component:  CSRF          |                  Version:  2.1
     Severity:  Normal        |               Resolution:  needsinfo
     Keywords:                |             Triage Stage:  Unreviewed
    Has patch:  0             |      Needs documentation:  0
  Needs tests:  0             |  Patch needs improvement:  0
Easy pickings:  0             |                    UI/UX:  0
------------------------------+--------------------------------------
Changes (by Rich Rauenzahn):

 * cc: Rich Rauenzahn (added)


--
Ticket URL: <https://code.djangoproject.com/ticket/30514#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.43c2e17c4e078b8e06d94af53682d375%40djangoproject.com.