problem with packages and path

Hello,

I'm writing some unit tests for my python software which uses
packages. Here is the basic structure:

mypackage
__init__.py
module1
__init__.py
mod1.py
module2
__init__.py
mod2.py
unittests
__init__.py
alltests.py
test1.py
test2.py

within alltests.py I would expect to be able to "import
mypackage.unittests.test1". In fact within PyScripter this works as
expected. However, when I execute the code from the command line, I
get the following error:

ImportError: No module named mypackage.unittests.test1

I've read that "When importing the package, Python searches through
the directories on sys.path looking for the package subdirectory." -
http://docs.python.org/tut/node8.htm...00000000000000

So I tried adding the following code to my alltests.py, which should
add the directory for mypackage to the path (and it does).

import os, sys
newpath = os.path.normpath( os.path.join( __file__, "../../" ))
sys.path.append(newpath)

I still get the same error. Can someone please point me in the right
direction? Thanks in advance:

Daniel

On 27 Aug, 18:44, Daniel <daniel.watr...@gmail.com> wrote:
>
> I'm writing some unit tests for my python software which uses
> packages. *Here is the basic structure:
>
> mypackage

[...]

> * unittests
> * * __init__.py
> * * alltests.py
> * * test1.py
> * * test2.py
>
> within alltests.py I would expect to be able to "import
> mypackage.unittests.test1". *In fact within PyScripter this works as
> expected. *However, when I execute the code from the command line, I
> get the following error:
>
> ImportError: No module named mypackage.unittests.test1

One thing to note: if you are running alltests.py directly (which you
don't mention explicitly) then even if you do so from the directory
containing the root of the package hierarchy (mypackage), it will be
the directory containing alltests.py (unittests) which will appear on
the PYTHONPATH/sys.path. Sometimes it's easy to take this behaviour
for granted when running programs.

A question: what happens if you prepend the directory containing the
root of package hierarchy to sys.path using insert (at element 0)
rather than append?

Paul

On Aug 27, 11:00*am, Paul Boddie <p...@boddie.org.uk> wrote:
> On 27 Aug, 18:44, Daniel <daniel.watr...@gmail.com> wrote:
>
>
>
> > I'm writing some unit tests for my python software which uses
> > packages. *Here is the basic structure:
>
> > mypackage
>
> [...]
>
> > * unittests
> > * * __init__.py
> > * * alltests.py
> > * * test1.py
> > * * test2.py
>
> > within alltests.py I would expect to be able to "import
> > mypackage.unittests.test1". *In fact within PyScripter this works as
> > expected. *However, when I execute the code from the command line, I
> > get the following error:
>
> > ImportError: No module named mypackage.unittests.test1
>
> One thing to note: if you are running alltests.py directly (which you
> don't mention explicitly) then even if you do so from the directory
> containing the root of the package hierarchy (mypackage), it will be
> the directory containing alltests.py (unittests) which will appear on
> the PYTHONPATH/sys.path. Sometimes it's easy to take this behaviour
> for granted when running programs.
>
> A question: what happens if you prepend the directory containing the
> root of package hierarchy to sys.path using insert (at element 0)
> rather than append?
>
> Paul

I changed it to 'sys.path.insert(0, newpath)', as you suggest, but it
doesn't fix the error.

I did notice that the directory containing the file alltests.py ends
up in the path. I've also noticed that (in WinXP at least) the
sys.path modification inserts a '..' in the path, rather than the full
path to 'mypackage'. I'm not sure if this makes a difference, but it
seems like I should be able to add either '..' or 'path/to/mypackage'
to the path and have it find my packages.

Any more suggestions?

On Aug 27, 11:00*am, Paul Boddie <p...@boddie.org.uk> wrote:
> On 27 Aug, 18:44, Daniel <daniel.watr...@gmail.com> wrote:
>
>
>
> > I'm writing some unit tests for my python software which uses
> > packages. *Here is the basic structure:
>
> > mypackage
>
> [...]
>
> > * unittests
> > * * __init__.py
> > * * alltests.py
> > * * test1.py
> > * * test2.py
>
> > within alltests.py I would expect to be able to "import
> > mypackage.unittests.test1". *In fact within PyScripter this works as
> > expected. *However, when I execute the code from the command line, I
> > get the following error:
>
> > ImportError: No module named mypackage.unittests.test1
>
> One thing to note: if you are running alltests.py directly (which you
> don't mention explicitly) then even if you do so from the directory
> containing the root of the package hierarchy (mypackage), it will be
> the directory containing alltests.py (unittests) which will appear on
> the PYTHONPATH/sys.path. Sometimes it's easy to take this behaviour
> for granted when running programs.
>
> A question: what happens if you prepend the directory containing the
> root of package hierarchy to sys.path using insert (at element 0)
> rather than append?
>
> Paul

I changed it to 'sys.path.insert(0, newpath)', as you suggest, but it
doesn't fix the error.

I did notice that the directory containing the file alltests.py ends
up in the path. I've also noticed that (in WinXP at least) the
sys.path modification inserts a '..' in the path, rather than the full
path to 'mypackage'. I'm not sure if this makes a difference, but it
seems like I should be able to add either '..' or 'path/to/mypackage'
to the path and have it find my packages.

Any more suggestions?

On Wed, Aug 27, 2008 at 6:44 PM, Daniel <daniel.watrous@gmail.com> wrote:
> Hello,
>
> I'm writing some unit tests for my python software which uses
> packages. Here is the basic structure:
>
> mypackage
> __init__.py
> module1
> __init__.py
> mod1.py
> module2
> __init__.py
> mod2.py
> unittests
> __init__.py
> alltests.py
> test1.py
> test2.py
>
> within alltests.py I would expect to be able to "import
> mypackage.unittests.test1". In fact within PyScripter this works as
> expected. However, when I execute the code from the command line, I
> get the following error:
>
> ImportError: No module named mypackage.unittests.test1
>

1) What is the command you're using to run the alltest.py module?

2) what is the result of:
- python -c "import mypackage"
- python -c "import mypackage.unittests"

Regards
Marco



--
Marco Bizzarri
http://iliveinpisa.blogspot.com/

On Aug 28, 2:28*am, "Marco Bizzarri" <marco.bizza...@gmail.com> wrote:
> On Wed, Aug 27, 2008 at 6:44 PM, Daniel <daniel.watr...@gmail.com> wrote:
> > Hello,
>
> > I'm writing some unit tests for my python software which uses
> > packages. *Here is the basic structure:
>
> > mypackage
> > *__init__.py
> > *module1
> > * *__init__.py
> > * *mod1.py
> > *module2
> > * *__init__.py
> > * *mod2.py
> > *unittests
> > * *__init__.py
> > * *alltests.py
> > * *test1.py
> > * *test2.py
>
> > within alltests.py I would expect to be able to "import
> > mypackage.unittests.test1". *In fact within PyScripter this works as
> > expected. *However, when I execute the code from the command line, I
> > get the following error:
>
> > ImportError: No module named mypackage.unittests.test1
>
> 1) What is the command you're using to run the alltest.py module?
>
> 2) what is the result of:
> * *- python -c "import mypackage"
> * - python -c "import mypackage.unittests"
>
> Regards
> Marco
>
> --
> Marco Bizzarrihttp://iliveinpisa.blogspot.com/

I have tried running both commands above from the mypackage directory
and unittests directory. I get the following response universtally.

C:\mypackage>dir
Volume in drive C is Default

Directory of C:\mypackage

08/29/2008 11:04 AM <DIR> .
08/29/2008 11:04 AM <DIR> ..
08/29/2008 11:05 AM <DIR> module1
08/29/2008 11:05 AM <DIR> module2
08/29/2008 11:06 AM <DIR> unittests
08/29/2008 11:04 AM 0 __init__.py
1 File(s) 0 bytes
5 Dir(s) 55,402,070,016 bytes free

C:\mypackage>dir unittests
Volume in drive C is Default

Directory of C:\mypackage\unittests

08/29/2008 11:06 AM <DIR> .
08/29/2008 11:06 AM <DIR> ..
08/29/2008 11:05 AM 0 alltests.py
08/29/2008 11:05 AM 0 test1.py
08/29/2008 11:05 AM 0 test2.py
08/29/2008 11:04 AM 0 __init__.py
4 File(s) 0 bytes
2 Dir(s) 55,401,988,096 bytes free

C:\mypackage>python -c "import mypackage"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named mypackage

C:\mypackage>python -c "import mypackage.unittests"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named mypackage.unittests

On 29 Aug, 19:08, Daniel <daniel.watr...@gmail.com> wrote:
>
> I have tried running both commands above from the mypackage directory
> and unittests directory. I get the following response universtally.
>
> C:\mypackage>dir
> Volume in drive C is Default
>
> Directory of C:\mypackage
>
> 08/29/2008 11:04 AM <DIR> .
> 08/29/2008 11:04 AM <DIR> ..
> 08/29/2008 11:05 AM <DIR> module1
> 08/29/2008 11:05 AM <DIR> module2
> 08/29/2008 11:06 AM <DIR> unittests
> 08/29/2008 11:04 AM 0 __init__.py
> 1 File(s) 0 bytes
> 5 Dir(s) 55,402,070,016 bytes free

If you run unittests\alltests.py from here, sys.path will initially
refer to this directory (C:\mypackage) and to anything else set up in
your environment. The extra path modification that you put in
alltests.py will append (or insert) the following...

os.path.normpath( os.path.join( __file__, "../../" ))
-> os.path.normpath( os.path.join( "C\\:mypackage\\unittests\
\alltests.py", "../../" ))
-> os.path.normpath( "C\\:mypackage" )
-> "C\\:mypackage"

Since the import statement then tries to access mypackage, which isn't
itself in the C:\mypackage directory - it's in the C:\ directory - the
import fails. The previous, automatically added entry in sys.path is
duplicated by the path modifications shown above, so that won't help
here, either.

> C:\mypackage>dir unittests
> Volume in drive C is Default
>
> Directory of C:\mypackage\unittests
>
> 08/29/2008 11:06 AM <DIR> .
> 08/29/2008 11:06 AM <DIR> ..
> 08/29/2008 11:05 AM 0 alltests.py
> 08/29/2008 11:05 AM 0 test1.py
> 08/29/2008 11:05 AM 0 test2.py
> 08/29/2008 11:04 AM 0 __init__.py
> 4 File(s) 0 bytes
> 2 Dir(s) 55,401,988,096 bytes free

Here, sys.path should refer to C:\mypackage\unittests somewhere, which
won't help Python find mypackage. The path modification in alltests.py
should perform as above, producing C:\mypackage which won't help.

I guess the easiest thing to do is to change the path modification
code to use "../../.." which should produce a reference to C:\ instead
of C:\mypackage. The confusion with this particular piece of the code
is the way ".." doesn't cause os.path.join to operate on the directory
C:\mypackage\unittests but instead on the file C:\mypackage\unittests
\alltests.py - that's almost to be expected, however, since
os.path.join doesn't really know that its first argument is really a
file or that ".." is being combined with a file.

What I tend to do, especially with __file__, is to first use
os.path.split to remove the filename - strictly speaking, it's the
leafname - and then to apply the kind of modifications mentioned
above, although there's probably no real difference or, indeed, any
advantage with either approach.

Paul

On Aug 29, 1:15*pm, Paul Boddie <p...@boddie.org.uk> wrote:
> On 29 Aug, 19:08, Daniel <daniel.watr...@gmail.com> wrote:
>
>
>
>
>
> > I have tried running both commands above from the mypackage directory
> > and unittests directory. *I get the following response universtally.
>
> > C:\mypackage>dir
> > *Volume in drive C is Default
>
> > *Directory of C:\mypackage
>
> > 08/29/2008 *11:04 AM * *<DIR> * * * * *.
> > 08/29/2008 *11:04 AM * *<DIR> * * * * *..
> > 08/29/2008 *11:05 AM * *<DIR> * * * * *module1
> > 08/29/2008 *11:05 AM * *<DIR> * * * * *module2
> > 08/29/2008 *11:06 AM * *<DIR> * * * * *unittests
> > 08/29/2008 *11:04 AM * * * * * * * * 0 __init__.py
> > * * * * * * * *1 File(s) * * * * * * *0 bytes
> > * * * * * * * *5 Dir(s) *55,402,070,016 bytes free
>
> If you run unittests\alltests.py from here, sys.pathwill initially
> refer to this directory (C:\mypackage) and to anything else set up in
> your environment. The extrapathmodification that you put in
> alltests.py will append (or insert) the following...
>
> * *os.path.normpath( os.path.join( __file__, "../../" ))
> -> os.path.normpath( os.path.join( "C\\:mypackage\\unittests\
> \alltests.py", "../../" ))
> -> os.path.normpath( "C\\:mypackage" )
> -> "C\\:mypackage"
>
> Since the import statement then tries to access mypackage, which isn't
> itself in the C:\mypackage directory - it's in the C:\ directory - the
> import fails. The previous, automatically added entry in sys.pathis
> duplicated by thepathmodifications shown above, so that won't help
> here, either.
>
> > C:\mypackage>dir unittests
> > *Volume in drive C is Default
>
> > *Directory of C:\mypackage\unittests
>
> > 08/29/2008 *11:06 AM * *<DIR> * * * * *.
> > 08/29/2008 *11:06 AM * *<DIR> * * * * *..
> > 08/29/2008 *11:05 AM * * * * * * * * 0 alltests.py
> > 08/29/2008 *11:05 AM * * * * * * * * 0 test1.py
> > 08/29/2008 *11:05 AM * * * * * * * * 0 test2.py
> > 08/29/2008 *11:04 AM * * * * * * * * 0 __init__.py
> > * * * * * * * *4 File(s) * * * * * * *0 bytes
> > * * * * * * * *2 Dir(s) *55,401,988,096 bytes free
>
> Here, sys.pathshould refer to C:\mypackage\unittests somewhere, which
> won't help Python find mypackage. Thepathmodification in alltests.py
> should perform as above, producing C:\mypackage which won't help.
>
> I guess the easiest thing to do is to change thepathmodification
> code to use "../../.." which should produce a reference to C:\ instead
> of C:\mypackage. The confusion with this particular piece of the code
> is the way ".." doesn't cause os.path.join to operate on the directory
> C:\mypackage\unittests but instead on the file C:\mypackage\unittests
> \alltests.py - that's almost to be expected, however, since
> os.path.join doesn't really know that its first argument is really a
> file or that ".." is being combined with a file.
>
> What I tend to do, especially with __file__, is to first use
> os.path.split to remove the filename - strictly speaking, it's the
> leafname - and then to apply the kind of modifications mentioned
> above, although there's probably no real difference or, indeed, any
> advantage with either approach.
>
> Paul

Thanks Paul. That did the trick.

Daniel