Quantcast

[issue14082] shutil doesn't support extended attributes

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't support extended attributes

STINNER Victor

New submission from Antoine Pitrou <[hidden email]>:

shutil doesn't provide any function which copies extended attributes on files. Note that "cp -a" does copy xattrs, but "cp -p" doesn't (and copy2() claims to work like "cp -p").

----------
components: Library (Lib)
messages: 153922
nosy: benjamin.peterson, hynek, neologix, pitrou
priority: low
severity: normal
status: open
title: shutil doesn't support extended attributes
type: enhancement
versions: Python 3.3

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't copy extended attributes

STINNER Victor

Éric Araujo <[hidden email]> added the comment:

> copy2() claims to work like "cp -p"
It probably does, for a behavior of “cp -p” that predates extended attributes <wink>.

Do you think the best way is to always copy xattrs, add a new parameter, add a new copy function?

----------
nosy: +eric.araujo
title: shutil doesn't support extended attributes -> shutil doesn't copy extended attributes

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't copy extended attributes

STINNER Victor
In reply to this post by STINNER Victor

Hynek Schlawack <[hidden email]> added the comment:

I'd tend to always copy xattrs – it seems that's what the user would expect to happen. A new parameter to _forbid_ it might make sense. However, I feel that there are already enough parameters in place. :-/

----------

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't copy extended attributes

STINNER Victor
In reply to this post by STINNER Victor

Charles-François Natali <[hidden email]> added the comment:

I'm also in favor of adding extended attributes to copy2:
"""
Similar to shutil.copy(), but metadata is copied as well
"""

extended attributes are metadata. And there are already too many copy functions...

----------

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't copy extended attributes

STINNER Victor
In reply to this post by STINNER Victor

Hynek Schlawack <[hidden email]> added the comment:

If nobody objects, I'd cook up a patch.

----------

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't copy extended attributes

STINNER Victor
In reply to this post by STINNER Victor

Éric Araujo <[hidden email]> added the comment:

Sounds good.

----------

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't copy extended attributes

STINNER Victor
In reply to this post by STINNER Victor

Changes by Arfrever Frehtes Taifersar Arahesis <[hidden email]>:


----------
nosy: +Arfrever

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't copy extended attributes

STINNER Victor
In reply to this post by STINNER Victor

Hynek Schlawack <[hidden email]> added the comment:

This ticket has a small catch:

There are several namespaces. According to http://en.wikipedia.org/wiki/Xattr#Linux :

- user: can be set by anyone
- trusted: root only
- security: root only
- system: even root can’t do that, at least not in my vm

I’m writing a shutil.copyxattr() first which could simple get another argument for the namespaces that should be copied.

However what to do inside copy2()?

I’m tending to either:

1. copy only user.*
2. ignore errors in any namespace != user

Personally, I find the second approach rather non-deterministic.

So I’d suggest:

- copyxattr has an extra argument called namespaces with default being ['user'], so that in theory someone who wants to do something more sophisticated can do it.
- copy2 copies only user.* though because that’s what you usually want.

Please let me know what you think about it.

----------

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't copy extended attributes

STINNER Victor

Antoine Pitrou <[hidden email]> added the comment:

> I’m writing a shutil.copyxattr() first which could simple get another
> argument for the namespaces that should be copied.

Sounds good to me :-)

> However what to do inside copy2()?
>
> I’m tending to either:
>
> 1. copy only user.*
> 2. ignore errors in any namespace != user
>
> Personally, I find the second approach rather non-deterministic.

But it's also more practical, e.g. when running as root you would
probably be surprised if only a subset of xattrs get copied, wouldn't
you? “Practicality beats purity.”

For reference, here is part of the documentation for GNU cp's "-a"
option:

`-a'
`--archive'
     Preserve as much as possible of the structure and attributes of the
     original files in the copy (but do not attempt to preserve internal
     directory structure; i.e., `ls -U' may list the entries in a copied
     directory in a different order).  Try to preserve SELinux security
     context and extended attributes (xattr), but ignore any failure to
     do that and print no corresponding diagnostic.  Equivalent to `-dR
     --preserve=all' with the reduced diagnostics.

Meaning that "cp -a" tries to copy all xattrs and silences errors when
it's not possible to do so.

"cp --preserve=all" seems to have a similar error-silencing behaviour:

    `all'
          Preserve all file attributes.  Equivalent to specifying all
          of the above, but with the difference that failure to
          preserve SELinux security context or extended attributes does
          not change `cp''s exit status.  In contrast to `-a', all but
          `Operation not supported' warnings are output.

----------

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't copy extended attributes

STINNER Victor
In reply to this post by STINNER Victor

Hynek Schlawack <[hidden email]> added the comment:

Ok, so I’ve added a function `copyxattr()` and `copy2()` tries to copy all possible namespaces.

Tests pass on Linux and Mac OS X.

----------
keywords: +patch
Added file: http://bugs.python.org/file25194/xattr.diff

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't copy extended attributes

STINNER Victor
In reply to this post by STINNER Victor

Changes by Hynek Schlawack <[hidden email]>:


----------
stage:  -> patch review

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't copy extended attributes

STINNER Victor
In reply to this post by STINNER Victor

Antoine Pitrou <[hidden email]> added the comment:

Hynek, did you get a notification of my review on Rietveld?

----------

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't copy extended attributes

STINNER Victor
In reply to this post by STINNER Victor

Hynek Schlawack <[hidden email]> added the comment:

I didn't. :/ I'll look into it tomorrow.

----------

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[issue14082] shutil doesn't copy extended attributes

STINNER Victor
In reply to this post by STINNER Victor

Hynek Schlawack <[hidden email]> added the comment:

I have answered to the (two weeks old :-/) review. There are three open questions in there we'll have to figure out before I fix the patch:

- should copyxattr() remove xattrs in dst that aren't present in src? Make it an option like `remove_missing_xattr`?

- use "None" for `namespaces` in copyxattrs() to indicate we want to copy all of them?

- add a ignore_errors option?

ISTM, that "all namespaces" don't make much sense without ignore_errors as there seem to be some internal xattr etc.

Suggestion: copyxattrs() has ignore_errors as default and returns a list of xattr it couldn't copy as (xattr, exception) tuples? Or an "on error" handler like in rmtree? I'd prefer the first one as ISTM that failures happen more often than not.

----------
assignee:  -> hynek

_______________________________________
Python tracker <[hidden email]>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/lists%2B1322467933539-512619%40n6.nabble.com

Loading...