Testing pre-release Django

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Testing pre-release Django

Claude Paroz
Hi,

I have the general feeling that too few people are testing the new Django major releases before the .0 release. The result being that many regressions are often reported after the release, while those could have been detected at alpha/beta/rc stages.

I found myself in the situation where I really wanted to test my own projects with a fresh new Django, but couldn't because some dependency was not updated and was crashing the project.

I wonder if it would help taking the most popular third-party dependencies and help those projects testing with pre-release Django. As an example, I added a Wiki page, https://code.djangoproject.com/wiki/Version1.10ThirdPartySupport (to be completed) as a mean to track Django 1.10 support progress in most popular apps. The first step would be to at least add Django 1.10 (or even master) as allowed_failures in Travis setups, so as 1.10 support is plainly visible.

Thoughts?

Claude

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/38f0a75a-dec3-46cd-b65a-1820594072df%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Testing pre-release Django

Tim Graham-2
I'm completely supportive of this effort. In past release cycles, I've done testing with djangoproject.com and sent some PRs to its dependencies. The blocker to upgrading is always waiting for each project to make a release with the fixes.

We could provide some guidance and suggest that projects should work on adding support for the next release of Django during the alpha period and try to make a next-Django-version compatible release by the beta release date, but of course we can't control how people spend their time.

I hoped that enforcing a feature freeze at alpha rather than beta (starting with the 1.8 release cycle) would make packages more eager to do prerelease testing since there should be less changes between then and the final release. I haven't seen evidence that this has helped though.

I'd be curious to hear from any third-party package maintainers how they feel about it.

On Friday, May 20, 2016 at 9:32:53 AM UTC-4, Claude Paroz wrote:
Hi,

I have the general feeling that too few people are testing the new Django major releases before the .0 release. The result being that many regressions are often reported after the release, while those could have been detected at alpha/beta/rc stages.

I found myself in the situation where I really wanted to test my own projects with a fresh new Django, but couldn't because some dependency was not updated and was crashing the project.

I wonder if it would help taking the most popular third-party dependencies and help those projects testing with pre-release Django. As an example, I added a Wiki page, <a href="https://code.djangoproject.com/wiki/Version1.10ThirdPartySupport" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fcode.djangoproject.com%2Fwiki%2FVersion1.10ThirdPartySupport\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFXLGkWxdxYCMDN6yWNlDLrazsI9g&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fcode.djangoproject.com%2Fwiki%2FVersion1.10ThirdPartySupport\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFXLGkWxdxYCMDN6yWNlDLrazsI9g&#39;;return true;">https://code.djangoproject.com/wiki/Version1.10ThirdPartySupport (to be completed) as a mean to track Django 1.10 support progress in most popular apps. The first step would be to at least add Django 1.10 (or even master) as allowed_failures in Travis setups, so as 1.10 support is plainly visible.

Thoughts?

Claude

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/c33bf5a9-70ca-4bd9-a4c0-13ad1654d69e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Testing pre-release Django

emorley
In reply to this post by Claude Paroz
Another idea might be to encourage more packages to test on Travis against Django master (with that sub-job marked as allowed to fail, so it doesn't fail the whole run) - so any incompatibilities become apparent earlier.

eg:
https://github.com/evansd/whitenoise/commit/c1a9f04cc90a7e48e536c651d9624660cee44073

On Friday, 20 May 2016 14:32:53 UTC+1, Claude Paroz wrote:
Hi,

I have the general feeling that too few people are testing the new Django major releases before the .0 release. The result being that many regressions are often reported after the release, while those could have been detected at alpha/beta/rc stages.

I found myself in the situation where I really wanted to test my own projects with a fresh new Django, but couldn't because some dependency was not updated and was crashing the project.

I wonder if it would help taking the most popular third-party dependencies and help those projects testing with pre-release Django. As an example, I added a Wiki page, <a href="https://code.djangoproject.com/wiki/Version1.10ThirdPartySupport" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fcode.djangoproject.com%2Fwiki%2FVersion1.10ThirdPartySupport\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFXLGkWxdxYCMDN6yWNlDLrazsI9g&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fcode.djangoproject.com%2Fwiki%2FVersion1.10ThirdPartySupport\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFXLGkWxdxYCMDN6yWNlDLrazsI9g&#39;;return true;">https://code.djangoproject.com/wiki/Version1.10ThirdPartySupport (to be completed) as a mean to track Django 1.10 support progress in most popular apps. The first step would be to at least add Django 1.10 (or even master) as allowed_failures in Travis setups, so as 1.10 support is plainly visible.

Thoughts?

Claude

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/7129d115-482e-4564-bce4-45322da2a0f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Testing pre-release Django

James Pic

Please test your projects against django master too.

On May 21, 2016 1:31 AM, "Ed Morley" <[hidden email]> wrote:
Another idea might be to encourage more packages to test on Travis against Django master (with that sub-job marked as allowed to fail, so it doesn't fail the whole run) - so any incompatibilities become apparent earlier.

eg:
https://github.com/evansd/whitenoise/commit/c1a9f04cc90a7e48e536c651d9624660cee44073

On Friday, 20 May 2016 14:32:53 UTC+1, Claude Paroz wrote:
Hi,

I have the general feeling that too few people are testing the new Django major releases before the .0 release. The result being that many regressions are often reported after the release, while those could have been detected at alpha/beta/rc stages.

I found myself in the situation where I really wanted to test my own projects with a fresh new Django, but couldn't because some dependency was not updated and was crashing the project.

I wonder if it would help taking the most popular third-party dependencies and help those projects testing with pre-release Django. As an example, I added a Wiki page, https://code.djangoproject.com/wiki/Version1.10ThirdPartySupport (to be completed) as a mean to track Django 1.10 support progress in most popular apps. The first step would be to at least add Django 1.10 (or even master) as allowed_failures in Travis setups, so as 1.10 support is plainly visible.

Thoughts?

Claude

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/7129d115-482e-4564-bce4-45322da2a0f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CALC3Kae1O8NZCqb3%3DuXZmGpD6Z74tOB8PG3bXA50sr4LgUhhJA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Testing pre-release Django

django-developers mailing list
In reply to this post by Tim Graham-2
When contributing to apps (dependencies), I've noticed developers tend to expect a stable django release before merging in support and making a new stable release, rather than doing so before the new django release.
This happened when I send patched to a few project with dj1.9 support.
 
If the django project provided some clear guidelines in the lines of "we recommend that you make your releases during the feature freeze period", with the explanations given in this thread,  it's be easier to convince those app developers to actually accept those patches and create those releases earlier, and start pushing the ecosystem in that direction.
 
The same applies to testing vs master.
 
Compatible [dependency] apps mean it's easier to test earlier too.
 
Just me two cents,
 
Cheers,
 
On Fri, May 20, 2016, at 14:50, Tim Graham wrote:
I'm completely supportive of this effort. In past release cycles, I've done testing with djangoproject.com and sent some PRs to its dependencies. The blocker to upgrading is always waiting for each project to make a release with the fixes.
 
We could provide some guidance and suggest that projects should work on adding support for the next release of Django during the alpha period and try to make a next-Django-version compatible release by the beta release date, but of course we can't control how people spend their time.
 
I hoped that enforcing a feature freeze at alpha rather than beta (starting with the 1.8 release cycle) would make packages more eager to do prerelease testing since there should be less changes between then and the final release. I haven't seen evidence that this has helped though.
 
I'd be curious to hear from any third-party package maintainers how they feel about it.
 
On Friday, May 20, 2016 at 9:32:53 AM UTC-4, Claude Paroz wrote:
Hi,
 
I have the general feeling that too few people are testing the new Django major releases before the .0 release. The result being that many regressions are often reported after the release, while those could have been detected at alpha/beta/rc stages.
 
I found myself in the situation where I really wanted to test my own projects with a fresh new Django, but couldn't because some dependency was not updated and was crashing the project.
 
I wonder if it would help taking the most popular third-party dependencies and help those projects testing with pre-release Django. As an example, I added a Wiki page, https://code.djangoproject.com/wiki/Version1.10ThirdPartySupport (to be completed) as a mean to track Django 1.10 support progress in most popular apps. The first step would be to at least add Django 1.10 (or even master) as allowed_failures in Travis setups, so as 1.10 support is plainly visible.
 
Thoughts?
 
Claude


--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
 
--
Hugo Osvaldo Barrera
 

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/1463817921.969445.614418497.0950C42C%40webmail.messagingengine.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Testing pre-release Django

tom christie
> I'd be curious to hear from any third-party package maintainers how they feel about it.

I've not found testing against master to be particularly helpful, as it's not apparent if an issue is substantial or not, without putting in work.

The alpha releases *are* tremendously helpful for having a pre-release to start working against,
although I don't know what a good process is for feeding back between third party packages that are updating, and the Django team.

For what it's worth, I'm seeing a surprising number of test failures for REST framework against 1.10a, although I suspect that may will be simple enough to resolve.

Another thing that might help is some guidance on ensuring that travis builds are properly displaying pending deprecation warnings - that'd go some way towards helping package maintainers get ready for a new release before it lands.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/65364570-3763-499f-a12b-f4102c5f1c6c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Testing pre-release Django

James Pic
I've found testing your own projects on django master to be
tremendously useful. Then, I don't have any surprise when I test on
django alpha, everything passes and I have nothing to do. Not to
mention the tremendous amount of things I learn on the way, at a
slower, more regular pace. Compare this to before, when I was waiting
too long to test against the new/next version of django, had to face a
ton of different failures at the last minute, had to go and couchsurf
at another contributor's house to peer-hack at night after our
dayjobs. BUT that's how I made a best friend for life and that's
priceless hehe even though I wouldn't hold this as a technical
argument for not testing against master !

I guess each person will have a different opinion. Nowadays, I add
djangomaster to my test matrix on all projects, in "allowed failures",
and find out it's broken every once in a while, and just have 1
problem to fix at the time. It works very well for our community, so,
definitely worth trying and then deciding if that's how you want to
work or not.

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CALC3KafR7NLJryntqAo7LkiBL3YMfk_yjLgs%3D2%3D_%2BJsnSTd_QQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Testing pre-release Django

Ola Sitarska
The way we do it for Djangae right now is creating a `support-110` branch whenever we've got some capacity to start working on it, or we wanna assess the amount of work required. Here is a current status for our Django 1.10 support.

However, we're aiming to move into making nightly builds on Travis that would include running tests on all supported Django versions + django master, once a day. It doesn't seem trivial though (seems like we will need to run our own version of https://nightli.es/ to run this build on different branch than master). Hopefully this will allow us to catch big breaking changes earlier, even before alpha is out. 

On Wed, May 25, 2016 at 11:57 AM, James Pic <[hidden email]> wrote:
I've found testing your own projects on django master to be
tremendously useful. Then, I don't have any surprise when I test on
django alpha, everything passes and I have nothing to do. Not to
mention the tremendous amount of things I learn on the way, at a
slower, more regular pace. Compare this to before, when I was waiting
too long to test against the new/next version of django, had to face a
ton of different failures at the last minute, had to go and couchsurf
at another contributor's house to peer-hack at night after our
dayjobs. BUT that's how I made a best friend for life and that's
priceless hehe even though I wouldn't hold this as a technical
argument for not testing against master !

I guess each person will have a different opinion. Nowadays, I add
djangomaster to my test matrix on all projects, in "allowed failures",
and find out it's broken every once in a while, and just have 1
problem to fix at the time. It works very well for our community, so,
definitely worth trying and then deciding if that's how you want to
work or not.

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CALC3KafR7NLJryntqAo7LkiBL3YMfk_yjLgs%3D2%3D_%2BJsnSTd_QQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFpnBuuq%2BeTCXV9bXW81%3Dv6ckOHEmsUKspx%2B9cbQEf_FE7uShw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.