A Heads Up for PyQt Packagers

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

A Heads Up for PyQt Packagers

Phil Thompson-5
This will mainly be of interest to people who create sip, PyQt4, PyQt5 etc. packages for Linux but also anybody who builds from source.

Starting with v5.11 (ie. the next release) PyQt5 will use a private copy of the sip module. Packagers have the choice of including it with PyQt5 or creating a new package (PyQt5_sip?). The wheels will use the second approach (because the module is Python version specific, whereas the PyQt5 modules themselves are not).

There will be one last release of PyQt4 which will also use a private copy of the sip module.

As a matter of interest, wxPython also uses a private copy of the sip module.

The change fixes a 20 year old design mistake and makes it much easier to move on with SIP v5 without worrying about the baggage of PyQt4.

Phil
_______________________________________________
PyQt mailing list    [hidden email]
https://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: A Heads Up for PyQt Packagers

Dmitry Shachnev
Hi Phil, and thanks for the heads up!

On Sun, Jun 17, 2018 at 10:27:54PM +0100, Phil Thompson wrote:
> This will mainly be of interest to people who create sip, PyQt4, PyQt5 etc.
> packages for Linux but also anybody who builds from source.
>
> Starting with v5.11 (ie. the next release) PyQt5 will use a private copy of
> the sip module. Packagers have the choice of including it with PyQt5 or
> creating a new package (PyQt5_sip?). The wheels will use the second approach
> (because the module is Python version specific, whereas the PyQt5 modules
> themselves are not).

What about “official” PyQt5 addons (such as QScintilla or PyQt3D), and
third-party PyQt5 addons — will they use standalone sip, or sip from PyQt5?

Will the standalone sip be updated, or 4.19.8 will be the last release of 4.x?

--
Dmitry Shachnev

_______________________________________________
PyQt mailing list    [hidden email]
https://www.riverbankcomputing.com/mailman/listinfo/pyqt

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: A Heads Up for PyQt Packagers

Phil Thompson-5
On 20 Jun 2018, at 5:35 pm, Dmitry Shachnev <[hidden email]> wrote:

>
> Hi Phil, and thanks for the heads up!
>
> On Sun, Jun 17, 2018 at 10:27:54PM +0100, Phil Thompson wrote:
>> This will mainly be of interest to people who create sip, PyQt4, PyQt5 etc.
>> packages for Linux but also anybody who builds from source.
>>
>> Starting with v5.11 (ie. the next release) PyQt5 will use a private copy of
>> the sip module. Packagers have the choice of including it with PyQt5 or
>> creating a new package (PyQt5_sip?). The wheels will use the second approach
>> (because the module is Python version specific, whereas the PyQt5 modules
>> themselves are not).
>
> What about “official” PyQt5 addons (such as QScintilla or PyQt3D), and
> third-party PyQt5 addons — will they use standalone sip, or sip from PyQt5?

They will all use the private copy.

> Will the standalone sip be updated, or 4.19.8 will be the last release of 4.x?


The creation of a private copy is just a configuration option for sip. I would expect Linux packages to continue to do exactly the same as before as far as sip is concerned. For PyQt5 the dependency on that sip package should be removed. The choice is then to create a new package containing PyQt5's private copy and make PyQt5 dependent on that - or just to include the private copy in the PyQt5 package itself.

Phil
_______________________________________________
PyQt mailing list    [hidden email]
https://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: A Heads Up for PyQt Packagers

Scott Kitterman
On Wednesday, June 20, 2018 10:08:35 PM Phil Thompson wrote:

> On 20 Jun 2018, at 5:35 pm, Dmitry Shachnev <[hidden email]> wrote:
> > Hi Phil, and thanks for the heads up!
> >
> > On Sun, Jun 17, 2018 at 10:27:54PM +0100, Phil Thompson wrote:
> >> This will mainly be of interest to people who create sip, PyQt4, PyQt5
> >> etc.
> >> packages for Linux but also anybody who builds from source.
> >>
> >> Starting with v5.11 (ie. the next release) PyQt5 will use a private copy
> >> of
> >> the sip module. Packagers have the choice of including it with PyQt5 or
> >> creating a new package (PyQt5_sip?). The wheels will use the second
> >> approach (because the module is Python version specific, whereas the
> >> PyQt5 modules themselves are not).
> >
> > What about “official” PyQt5 addons (such as QScintilla or PyQt3D), and
> > third-party PyQt5 addons — will they use standalone sip, or sip from
> > PyQt5?
>
> They will all use the private copy.
>
> > Will the standalone sip be updated, or 4.19.8 will be the last release of
> > 4.x?
> The creation of a private copy is just a configuration option for sip. I
> would expect Linux packages to continue to do exactly the same as before as
> far as sip is concerned. For PyQt5 the dependency on that sip package
> should be removed. The choice is then to create a new package containing
> PyQt5's private copy and make PyQt5 dependent on that - or just to include
> the private copy in the PyQt5 package itself.

If we make a package for the 'private copy' can QScintilla2, etc use the same
one?

Scott K
_______________________________________________
PyQt mailing list    [hidden email]
https://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: A Heads Up for PyQt Packagers

Phil Thompson-5
On 20 Jun 2018, at 10:21 pm, Scott Kitterman <[hidden email]> wrote:

>
> On Wednesday, June 20, 2018 10:08:35 PM Phil Thompson wrote:
>> On 20 Jun 2018, at 5:35 pm, Dmitry Shachnev <[hidden email]> wrote:
>>> Hi Phil, and thanks for the heads up!
>>>
>>> On Sun, Jun 17, 2018 at 10:27:54PM +0100, Phil Thompson wrote:
>>>> This will mainly be of interest to people who create sip, PyQt4, PyQt5
>>>> etc.
>>>> packages for Linux but also anybody who builds from source.
>>>>
>>>> Starting with v5.11 (ie. the next release) PyQt5 will use a private copy
>>>> of
>>>> the sip module. Packagers have the choice of including it with PyQt5 or
>>>> creating a new package (PyQt5_sip?). The wheels will use the second
>>>> approach (because the module is Python version specific, whereas the
>>>> PyQt5 modules themselves are not).
>>>
>>> What about “official” PyQt5 addons (such as QScintilla or PyQt3D), and
>>> third-party PyQt5 addons — will they use standalone sip, or sip from
>>> PyQt5?
>>
>> They will all use the private copy.
>>
>>> Will the standalone sip be updated, or 4.19.8 will be the last release of
>>> 4.x?
>> The creation of a private copy is just a configuration option for sip. I
>> would expect Linux packages to continue to do exactly the same as before as
>> far as sip is concerned. For PyQt5 the dependency on that sip package
>> should be removed. The choice is then to create a new package containing
>> PyQt5's private copy and make PyQt5 dependent on that - or just to include
>> the private copy in the PyQt5 package itself.
>
> If we make a package for the 'private copy' can QScintilla2, etc use the same
> one?

There is no need. The configuration and build of "sub-packages" like that will automatically pick up the private copy.

Phil
_______________________________________________
PyQt mailing list    [hidden email]
https://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: A Heads Up for PyQt Packagers

Dmitry Shachnev
On Thu, Jun 21, 2018 at 09:12:19AM +0100, Phil Thompson wrote:
> > If we make a package for the 'private copy' can QScintilla2, etc use
> > the same one?
>
> There is no need. The configuration and build of "sub-packages" like
> that will automatically pick up the private copy.

Thanks for your replies, I hope we’ll figure out how to deal with the
new versions once they are released.

--
Dmitry Shachnev
_______________________________________________
PyQt mailing list    [hidden email]
https://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: A Heads Up for PyQt Packagers

Florian Bruhin
In reply to this post by Phil Thompson-5
On Sun, Jun 17, 2018 at 10:27:54PM +0100, Phil Thompson wrote:
> This will mainly be of interest to people who create sip, PyQt4, PyQt5
> etc. packages for Linux but also anybody who builds from source.
>
> Starting with v5.11 (ie. the next release) PyQt5 will use a private
> copy of the sip module. Packagers have the choice of including it with
> PyQt5 or creating a new package (PyQt5_sip?). The wheels will use the
> second approach (because the module is Python version specific,
> whereas the PyQt5 modules themselves are not).

Let me note that this also requires projects to adjust their imports to
continue working with the latest wheels - instead of "import sip", one
now needs to do "from PyQt5 import sip".

Florian

--
https://www.qutebrowser.org | [hidden email] (Mail/XMPP)
   GPG: 916E B0C8 FD55 A072 | https://the-compiler.org/pubkey.asc
         I love long mails! | https://email.is-not-s.ms/

_______________________________________________
PyQt mailing list    [hidden email]
https://www.riverbankcomputing.com/mailman/listinfo/pyqt

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: A Heads Up for PyQt Packagers

Phil Thompson-5
On 23 Jun 2018, at 4:15 pm, Florian Bruhin <[hidden email]> wrote:

>
> On Sun, Jun 17, 2018 at 10:27:54PM +0100, Phil Thompson wrote:
>> This will mainly be of interest to people who create sip, PyQt4, PyQt5
>> etc. packages for Linux but also anybody who builds from source.
>>
>> Starting with v5.11 (ie. the next release) PyQt5 will use a private
>> copy of the sip module. Packagers have the choice of including it with
>> PyQt5 or creating a new package (PyQt5_sip?). The wheels will use the
>> second approach (because the module is Python version specific,
>> whereas the PyQt5 modules themselves are not).
>
> Let me note that this also requires projects to adjust their imports to
> continue working with the latest wheels - instead of "import sip", one
> now needs to do "from PyQt5 import sip".

http://pyqt.sourceforge.net/Docs/PyQt5/incompatibilities.html#pyqt-v5-11

Phil
_______________________________________________
PyQt mailing list    [hidden email]
https://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: A Heads Up for PyQt Packagers

Kovid Goyal
In reply to this post by Phil Thompson-5
How is one supposed to build one's own PyQt extensions. Currently I do
that by calling the system wide sip binary to process the SIP files. I
assume I now have to use the sip binary bundled with PyQt? Is there one?
And if so, how do I get the path to it?

Kovid.

On Sun, Jun 17, 2018 at 10:27:54PM +0100, Phil Thompson wrote:

> This will mainly be of interest to people who create sip, PyQt4, PyQt5 etc. packages for Linux but also anybody who builds from source.
>
> Starting with v5.11 (ie. the next release) PyQt5 will use a private copy of the sip module. Packagers have the choice of including it with PyQt5 or creating a new package (PyQt5_sip?). The wheels will use the second approach (because the module is Python version specific, whereas the PyQt5 modules themselves are not).
>
> There will be one last release of PyQt4 which will also use a private copy of the sip module.
>
> As a matter of interest, wxPython also uses a private copy of the sip module.
>
> The change fixes a 20 year old design mistake and makes it much easier to move on with SIP v5 without worrying about the baggage of PyQt4.
>
> Phil
> _______________________________________________
> PyQt mailing list    [hidden email]
> https://www.riverbankcomputing.com/mailman/listinfo/pyqt

--
_____________________________________

Dr. Kovid Goyal
https://www.kovidgoyal.net
https://calibre-ebook.com
_____________________________________
_______________________________________________
PyQt mailing list    [hidden email]
https://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: A Heads Up for PyQt Packagers

Phil Thompson-5
On 24 Jun 2018, at 3:55 am, Kovid Goyal <[hidden email]> wrote:
>
> How is one supposed to build one's own PyQt extensions. Currently I do
> that by calling the system wide sip binary to process the SIP files. I
> assume I now have to use the sip binary bundled with PyQt? Is there one?
> And if so, how do I get the path to it?

There is no private copy of the code generator, just an extra flag to generate a private copy of the module. See...

http://pyqt.sourceforge.net/Docs/sip4/using.html#building-a-private-copy-of-the-sip-module

Phil
_______________________________________________
PyQt mailing list    [hidden email]
https://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: A Heads Up for PyQt Packagers

Hans-Peter Jansen-2
In reply to this post by Phil Thompson-5
Dear Phil,

On Sonntag, 17. Juni 2018 22:27:54 Phil Thompson wrote:

> This will mainly be of interest to people who create sip, PyQt4, PyQt5 etc.
> packages for Linux but also anybody who builds from source.
>
> Starting with v5.11 (ie. the next release) PyQt5 will use a private copy of
> the sip module. Packagers have the choice of including it with PyQt5 or
> creating a new package (PyQt5_sip?). The wheels will use the second
> approach (because the module is Python version specific, whereas the PyQt5
> modules themselves are not).
>
> There will be one last release of PyQt4 which will also use a private copy
> of the sip module.
>
> As a matter of interest, wxPython also uses a private copy of the sip
> module.
>
> The change fixes a 20 year old design mistake and makes it much easier to
> move on with SIP v5 without worrying about the baggage of PyQt4.

While this might ease further sip development, it will create headaches for
sip users for what I can imagine, since this will lead to a sip version
inflation over time on certain systems.

Attending sip and pyqt development for about 20 years now, I can see, that  
external backflow to sip development is pretty low unfortunately. On the other
hand, it has *always* been a pleasure to package your stuff and to work with.

>From a packager POV, ending with sip4_qt4, sip4_qt5, sip4_kde4, sip4_kde5,
sip4_mypackage isn't exactly funny. Adding sip5 to the mix will further add
jags to the picture.

If a developer want to use the sip technology, first step would be choosing
the relevant sip version, clone it, and stick with it. Further improvements to
sip will most probably pass by since there's no pressure to get it right for
all sip users.

Given the scale of the generated bindings, I fear, that issues may pass
unnoticed with the new model much longer than now. This problem could be
combated with automatic generation of (python) test code, but this is a
mammoth project, that isn't going to be realized with the available manpower
any time soon. With the KDE bindings in mind, I bet, that there are a lot of
code paths, that were never exercised, nowhere. With my personal rule of thump
"code paths, that aren't exercised, are broken", this leaves behind an
uncomfortable feeling in this regard.

Hence, I'm not sure, if sip diversity will pay off in the end.

Please don't get me wrong, but branching and unitizing might be viable
solutions to the problem, that also reduce code diversity.

Cheers,
Pete
_______________________________________________
PyQt mailing list    [hidden email]
https://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: A Heads Up for PyQt Packagers

Phil Thompson-5
On 25 Jun 2018, at 12:42 pm, Hans-Peter Jansen <[hidden email]> wrote:

>
> Dear Phil,
>
> On Sonntag, 17. Juni 2018 22:27:54 Phil Thompson wrote:
>> This will mainly be of interest to people who create sip, PyQt4, PyQt5 etc.
>> packages for Linux but also anybody who builds from source.
>>
>> Starting with v5.11 (ie. the next release) PyQt5 will use a private copy of
>> the sip module. Packagers have the choice of including it with PyQt5 or
>> creating a new package (PyQt5_sip?). The wheels will use the second
>> approach (because the module is Python version specific, whereas the PyQt5
>> modules themselves are not).
>>
>> There will be one last release of PyQt4 which will also use a private copy
>> of the sip module.
>>
>> As a matter of interest, wxPython also uses a private copy of the sip
>> module.
>>
>> The change fixes a 20 year old design mistake and makes it much easier to
>> move on with SIP v5 without worrying about the baggage of PyQt4.
>
> While this might ease further sip development, it will create headaches for
> sip users for what I can imagine, since this will lead to a sip version
> inflation over time on certain systems.
>
> Attending sip and pyqt development for about 20 years now, I can see, that  
> external backflow to sip development is pretty low unfortunately. On the other
> hand, it has *always* been a pleasure to package your stuff and to work with.

> From a packager POV, ending with sip4_qt4, sip4_qt5, sip4_kde4, sip4_kde5,
> sip4_mypackage isn't exactly funny. Adding sip5 to the mix will further add
> jags to the picture.
>
> If a developer want to use the sip technology, first step would be choosing
> the relevant sip version, clone it, and stick with it. Further improvements to
> sip will most probably pass by since there's no pressure to get it right for
> all sip users.
>
> Given the scale of the generated bindings, I fear, that issues may pass
> unnoticed with the new model much longer than now. This problem could be
> combated with automatic generation of (python) test code, but this is a
> mammoth project, that isn't going to be realized with the available manpower
> any time soon. With the KDE bindings in mind, I bet, that there are a lot of
> code paths, that were never exercised, nowhere. With my personal rule of thump
> "code paths, that aren't exercised, are broken", this leaves behind an
> uncomfortable feeling in this regard.
>
> Hence, I'm not sure, if sip diversity will pay off in the end.
>
> Please don't get me wrong, but branching and unitizing might be viable
> solutions to the problem, that also reduce code diversity.

Whether or not you create new packages for the private copy of the sip module is entirely down to you as a packager. If it were me I would do it so that the number and names of the packages would be exactly the same as they are today. Only the dependencies between them would change. Specifically I would include the private copy of the sip module in my PyQt4 and PyQt5 packages. The dependency of your PyQt4/PyQt5 packages on your SIP package would be removed. Whether the private copy was created by SIP4 or SIP5 is irrelevant. Any kde4 or kde5 would not include a private copy of the sip module - they would automatically use PyQt's copy.

When it comes to wheels I take a different approach and package the private copy of the sip module separately, ie. it has its own wheel. The reason for this is that the sip module is dependent on a specific version of Python (3.5, 3.6, 3.7 etc) but presents a version-independent API to PyQt etc. Therefore to add support for (say) Python v3.8 I just need to release a new sip module wheel for that version and the existing PyQt wheels will automatically work.

I think one cause of confusion is my use of the word "private". PyQt and PyQt3D and QScintilla etc, etc form a hierachy of modules rooted at the QtCore module. All of those modules share the same "private" copy of the sip module. It is perfectly reasonable for me (as the author of PyQt) to place restrictions on the version of the sip module shared by all those modules (including any written by other people). However it's not reasonable for those restrictions to be forced onto authors of other modules that have nothing to do with PyQt.

Phil
_______________________________________________
PyQt mailing list    [hidden email]
https://www.riverbankcomputing.com/mailman/listinfo/pyqt