unexpected from/import statement behaviour

Hi all !

I'm trying to capture stderr of an external module I use in my python
program. I'm doing this
by setting up a class in my module overwriting the stderr file object
method write.
The external module outputs to stderr this way:

from sys import std err

.....

print >> stderr, "Some text"

While in my module I use

import sys

..... sys.stderr ... sys.stdout

Well, as long as I do not change in the external module those from/
import statements to just

import sys

.....

print >> sys.stderr, "Some text"

I'm not able to capture its stderr and of course I would like not to
do this kind of change.
I've always been convinced of the equivalence of the two ways of using
the import statement
but it's clear I'm wrong :-(

Please, someone can tell me what's going on ?

Thanks in advance !

nisp schrieb:
> Hi all !
>
> I'm trying to capture stderr of an external module I use in my python
> program. I'm doing this
> by setting up a class in my module overwriting the stderr file object
> method write.
> The external module outputs to stderr this way:
>
> from sys import std err
>
> ....
>
> print >> stderr, "Some text"
>
> While in my module I use
>
> import sys
>
> ..... sys.stderr ... sys.stdout
>
> Well, as long as I do not change in the external module those from/
> import statements to just
>
> import sys
>
> ....
>
> print >> sys.stderr, "Some text"
>
> I'm not able to capture its stderr and of course I would like not to
> do this kind of change.
> I've always been convinced of the equivalence of the two ways of using
> the import statement
> but it's clear I'm wrong :-(
>
> Please, someone can tell me what's going on ?

http://effbot.org/zone/import-confusion.htm

Diez

nisp <emanuele.nesp...@gmail.com> wrote:
> I've always been convinced of the equivalence of the two ways of using
> the import statement but it's clear I'm wrong :-(

Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from sys import stderr
>>> import sys
>>> sys.stderr is stderr
True

Behaviourly, I'm finding no difference between the two either.

Could you cut & paste a minimal example that isn't producing the
correct behaviour, and perhaps mention what type of OS you're using?

alex23 schrieb:
> nisp <emanuele.nesp...@gmail.com> wrote:
>> I've always been convinced of the equivalence of the two ways of using
>> the import statement but it's clear I'm wrong :-(
>
> Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
> [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from sys import stderr
>>>> import sys
>>>> sys.stderr is stderr
> True
>
> Behaviourly, I'm finding no difference between the two either.
>
> Could you cut & paste a minimal example that isn't producing the
> correct behaviour, and perhaps mention what type of OS you're using?

You did not read this part of the post:

"""
I'm not able to capture its stderr and of course I would like not to
do this kind of change.
"""

He tries to set sys.stderr to a new stream and capture the print
statements, but fails to do so because he created a local alias.


Diez

nisp wrote:

> Hi all !
>
> I'm trying to capture stderr of an external module I use in my python
> program. I'm doing this
> by setting up a class in my module overwriting the stderr file object
> method write.
> The external module outputs to stderr this way:
>
> from sys import std err
>
> ....
>
> print >> stderr, "Some text"
>
> While in my module I use
>
> import sys
>
> ..... sys.stderr ... sys.stdout
>
> Well, as long as I do not change in the external module those from/
> import statements to just
>
> import sys
>
> ....
>
> print >> sys.stderr, "Some text"
>
> I'm not able to capture its stderr and of course I would like not to
> do this kind of change.
> I've always been convinced of the equivalence of the two ways of using
> the import statement
> but it's clear I'm wrong :-(
>
> Please, someone can tell me what's going on ?
>
> Thanks in advance !

A practical approach to complement Diez' link to the explanation:

Instead of modifying the external module you can either redirect stderr
before you import the external module

import sys
sys.stderr = whatever
import external

or monkey-patch:

import sys
import external

sys.stderr = external.sterr = whatever

Peter

On Aug 27, 9:56 am, Peter Otten <__pete...@web.de> wrote:
> nisp wrote:
> > Hi all !
>
> > I'm trying to capture stderr of an external module I use in my python
> > program. I'm doing this
> > by setting up a class in my module overwriting the stderr file object
> > method write.
> > The external module outputs to stderr this way:
>
> > from sys import std err
>
> > ....
>
> > print >> stderr, "Some text"
>
> > While in my module I use
>
> > import sys
>
> > ..... sys.stderr ... sys.stdout
>
> > Well, as long as I do not change in the external module those from/
> > import statements to just
>
> > import sys
>
> > ....
>
> > print >> sys.stderr, "Some text"
>
> > I'm not able to capture its stderr and of course I would like not to
> > do this kind of change.
> > I've always been convinced of the equivalence of the two ways of using
> > the import statement
> > but it's clear I'm wrong :-(
>
> > Please, someone can tell me what's going on ?
>
> > Thanks in advance !
>
> A practical approach to complement Diez' link to the explanation:
>
> Instead of modifying the external module you can either redirect stderr
> before you import the external module
>
> import sys
> sys.stderr = whatever
> import external
>
> or monkey-patch:
>
> import sys
> import external
>
> sys.stderr = external.sterr = whatever
>
> Peter

Hi all !

Thanks first of all ! I read the interesting Diez's link but something
still remains to me unclear, on the other hand it's clear the my
problem is concentrated there and on symbols.
Here is what I'm trying to do

HelloWorld.py: this is a real simplification of my external module
though still reflecting its structure (commented out is the version
that, let's say, works)

from sys import stderr
#import sys

class Cheers:
def __init__(self):
self.cheersMsg = 'Hello World !!'
print "Cheers stderr %s" % stderr
#print "Cheers stderr %s" % sys.stderr
def _putCheers(self):
print>>stderr, 'Here is my msg:', self.cheersMsg
print>>stderr, 'This is a nice day today !!'
#print>>sys.stderr, 'Here is my msg:', self.cheersMsg
#print>>sys.stderr, 'This is a nice day today !!'
def doSomeStuff(self):
self._putCheers()

And below there is the module that uses the above one (mymodule.py):

#!/usr/bin/python

import sys
from HelloWorld import *

class StderrCatcher:
def __init__(self):
self.data = ''
def write(self,stuff):
self.data = self.data + "\t" + stuff


def main():

print "mymodule stderr: %s" % sys.stderr

sys.stderr = stderr = StderrCatcher()
m = Cheers()
m.doSomeStuff()
print "stderr: \n%s" % sys.stderr.data

if __name__ == '__main__':
main()


Below there is the output when it doesn't work:

mymodule stderr: <open file '<stderr>', mode 'w' at 0xb7d160b0>
Cheers stderr <open file '<stderr>', mode 'w' at 0xb7d160b0>
Here is my msg: Hello World !!
This is a nice day today !!
stderr:

And here when it works:

mymodule stderr: <open file '<stderr>', mode 'w' at 0xb7dc40b0>
Cheers stderr <__main__.StderrCatcher instance at 0xb7d8bd4c>
stderr:
Here is my msg: Hello World !!
This is a nice day today !!


Thanks again!

PS Sorry for having probably replied to somone of you directly :-(

nisp wrote:

> On Aug 27, 9:56 am, Peter Otten <__pete...@web.de> wrote:
>> nisp wrote:
>> > Hi all !
>>
>> > I'm trying to capture stderr of an external module I use in my python
>> > program. I'm doing this
>> > by setting up a class in my module overwriting the stderr file object
>> > method write.
>> > The external module outputs to stderr this way:
>>
>> > from sys import std err
>>
>> > ....
>>
>> > print >> stderr, "Some text"
>>
>> > While in my module I use
>>
>> > import sys
>>
>> > ..... sys.stderr ... sys.stdout
>>
>> > Well, as long as I do not change in the external module those from/
>> > import statements to just
>>
>> > import sys
>>
>> > ....
>>
>> > print >> sys.stderr, "Some text"
>>
>> > I'm not able to capture its stderr and of course I would like not to
>> > do this kind of change.
>> > I've always been convinced of the equivalence of the two ways of using
>> > the import statement
>> > but it's clear I'm wrong :-(
>>
>> > Please, someone can tell me what's going on ?
>>
>> > Thanks in advance !
>>
>> A practical approach to complement Diez' link to the explanation:
>>
>> Instead of modifying the external module you can either redirect stderr
>> before you import the external module
>>
>> import sys
>> sys.stderr = whatever
>> import external
>>
>> or monkey-patch:
>>
>> import sys
>> import external
>>
>> sys.stderr = external.sterr = whatever
>>
>> Peter
>
> Hi all !
>
> Thanks first of all ! I read the interesting Diez's link but something
> still remains to me unclear, on the other hand it's clear the my
> problem is concentrated there and on symbols.
> Here is what I'm trying to do
>
> HelloWorld.py: this is a real simplification of my external module
> though still reflecting its structure (commented out is the version
> that, let's say, works)
>
> from sys import stderr

This is your problem. You create a HelloWorld.stderr-alias to the object
bound to sys.stderr. Rebinding the latter won't affect the former. That is
precisely what the link I gave you explains.

The short answer to the whole issue is: dont' use the from-import syntax
until you really know what you are doing. Or not at all.

Diez

On Aug 27, 2:43 pm, "Diez B. Roggisch" <de...@nospam.web.de> wrote:
> nisp wrote:
> > On Aug 27, 9:56 am, Peter Otten <__pete...@web.de> wrote:
> >> nisp wrote:
> >> > Hi all !
>
> >> > I'm trying to capture stderr of an external module I use in my python
> >> > program. I'm doing this
> >> > by setting up a class in my module overwriting the stderr file object
> >> > method write.
> >> > The external module outputs to stderr this way:
>
> >> > from sys import std err
>
> >> > ....
>
> >> > print >> stderr, "Some text"
>
> >> > While in my module I use
>
> >> > import sys
>
> >> > ..... sys.stderr ... sys.stdout
>
> >> > Well, as long as I do not change in the external module those from/
> >> > import statements to just
>
> >> > import sys
>
> >> > ....
>
> >> > print >> sys.stderr, "Some text"
>
> >> > I'm not able to capture its stderr and of course I would like not to
> >> > do this kind of change.
> >> > I've always been convinced of the equivalence of the two ways of using
> >> > the import statement
> >> > but it's clear I'm wrong :-(
>
> >> > Please, someone can tell me what's going on ?
>
> >> > Thanks in advance !
>
> >> A practical approach to complement Diez' link to the explanation:
>
> >> Instead of modifying the external module you can either redirect stderr
> >> before you import the external module
>
> >> import sys
> >> sys.stderr = whatever
> >> import external
>
> >> or monkey-patch:
>
> >> import sys
> >> import external
>
> >> sys.stderr = external.sterr = whatever
>
> >> Peter
>
> > Hi all !
>
> > Thanks first of all ! I read the interesting Diez's link but something
> > still remains to me unclear, on the other hand it's clear the my
> > problem is concentrated there and on symbols.
> > Here is what I'm trying to do
>
> > HelloWorld.py: this is a real simplification of my external module
> > though still reflecting its structure (commented out is the version
> > that, let's say, works)
>
> > from sys import stderr
>
> This is your problem. You create a HelloWorld.stderr-alias to the object
> bound to sys.stderr. Rebinding the latter won't affect the former. That is
> precisely what the link I gave you explains.
>
> The short answer to the whole issue is: dont' use the from-import syntax
> until you really know what you are doing. Or not at all.
>
> Diez

Hi Diez! I well understand the problem now, the fact is that
unfortunately the external module is not mine and i doubt i can change
it :-( Anyway as far as i'm concerned i've learned the lesson ! Thanks
again and greetings,
Nisp

nisp wrote:

> Thanks first of all ! I read the interesting Diez's link but something
> still remains to me unclear, on the other hand it's clear the my
> problem is concentrated there and on symbols.

Read it again. If you have two modules

module1.py
from sys import stderr

module2.py
from module1 import stderr

you get three names 'stderr' in three different namespaces, and each name
may bind a different object.


> Here is what I'm trying to do
>
> HelloWorld.py: this is a real simplification of my external module
> though still reflecting its structure (commented out is the version
> that, let's say, works)
>
> from sys import stderr
> #import sys
>
> class Cheers:
> Â*Â*Â*Â*Â*Â*Â*Â*def __init__(self):
> Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*self.cheersMsg = 'Hello World !!'
> Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*print "Cheers stderr %s" % stderr
> Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*#print "Cheers stderr %s" % sys.stderr
> Â*Â*Â*Â*Â*Â*Â*Â*def _putCheers(self):
> Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*print>>stderr, 'Here is my msg:', self.cheersMsg
> Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*print>>stderr, 'This is a nice day today !!'
> Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*#print>>sys.stderr , 'Here is my msg:', self.cheersMsg
> Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*#print>>sys.stderr , 'This is a nice day today !!'
> Â*Â*Â*Â*Â*Â*Â*Â*def doSomeStuff(self):
> Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*self._putCheers()
>
> And below there is the module that uses the above one (mymodule.py):
>
> #!/usr/bin/python
>
> import sys

This imports HelloWorld.stderr:

> from HelloWorld import *

You now have a global variable 'stderr' in module __main__, initialized to
the same value as HelloWorld.stderr.

> class StderrCatcher:
> Â*Â*Â*Â*Â*Â*Â*Â*def __init__(self):
> Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*self.data = ''
> Â*Â*Â*Â*Â*Â*Â*Â*def write(self,stuff):
> Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*self.data = self.data + "\t" + stuff
>
>
> def main():
>
> Â*Â*Â*Â*Â*Â*Â*Â*print "mymodule stderr: %s" % sys.stderr
>

This rebinds sys.stderr and a local 'stderr' in main()

> Â*Â*Â*Â*Â*Â*Â*Â*sys.stderr = stderr = StderrCatcher()

but both HelloWorld.stderr and __main__.stderr are unaffected. What you need
is
import HelloWorld
sys.stderr = HelloWorld.stderr = StderrCatcher()

> Â*Â*Â*Â*Â*Â*Â*Â*m = Cheers()
> Â*Â*Â*Â*Â*Â*Â*Â*m.doSomeStuff()
> Â*Â*Â*Â*Â*Â*Â*Â*print "stderr: \n%s" % sys.stderr.data
>
> if __name__ == '__main__':
> Â* Â* main()
>
>
> Below there is the output when it doesn't work:
>
> mymodule stderr: <open file '<stderr>', mode 'w' at 0xb7d160b0>
> Cheers stderr <open file '<stderr>', mode 'w' at 0xb7d160b0>
> Here is my msg: Hello World !!
> This is a nice day today !!
> stderr:
>
> And here when it works:
>
> mymodule stderr: <open file '<stderr>', mode 'w' at 0xb7dc40b0>
> Cheers stderr <__main__.StderrCatcher instance at 0xb7d8bd4c>
> stderr:
> Â* Â* Â* Â* Here is my msg: Â* Â* Â* Â* Hello World !!
> Â* Â* Â* Â* This is a nice day today !!
>
>
> Thanks again!
>
> PS Sorry for having probably replied to somone of you directly :-(

On Aug 27, 5:42*pm, "Diez B. Roggisch" <de...@nospam.web.de> wrote:
> alex23 schrieb:
>
>
>
> > nisp <emanuele.nesp...@gmail.com> wrote:
> >> I've always been convinced of the equivalence of the two ways of using
> >> the import statement but it's clear I'm wrong :-(
>
> > Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
> > [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> >>>> from sys import stderr
> >>>> import sys
> >>>> sys.stderr is stderr
> > True
>
> > Behaviourly, I'm finding no difference between the two either.
>
> > Could you cut & paste a minimal example that isn't producing the
> > correct behaviour, and perhaps mention what type of OS you're using?
>
> You did not read this part of the post:
>
> """
> I'm not able to capture its stderr and of course I would like not to
> do this kind of change.
> """
>
> He tries to set sys.stderr to a new stream and capture the print
> statements, but fails to do so because he created a local alias.

No, I read it, I just totally didn't parse it that way

Sorry for the confusion, nisp.