Where to locate existing standard encodings in python - Python

On Nov 10, 11:00 am, News123 <news...@free.fr> wrote:
> Hi,
>
> I was googling quite some time before finding the answer to my question:
> 'what are the names for the encodings supported by python?'
>
> I found the answer athttp://python.active-venture.com/lib/node127.html
>
> Now my question:
>
> Can I find the same info in the standard python doc or query python with
> a certain command to print out all existing codings?

codecs module
http://docs.python.org/library/codecs.html#id3

> I was googling quite some time before finding the answer to my question:
> 'what are the names for the encodings supported by python?'
>
> I found the answer at http://python.active-venture.com/lib/node127.html
>
> Now my question:
>
> Can I find the same info in the standard python doc or query python with
> a certain command to print out all existing codings?

http://www.python.org/doc/2.5.2/lib/...encodings.html

-tkc

On Nov 9, 2008, at 7:00 PM, News123 wrote:

> Hi,
>
> I was googling quite some time before finding the answer to my
> question:
> 'what are the names for the encodings supported by python?'
>
> I found the answer at http://python.active-venture.com/lib/
> node127.html
>
>
> Now my question:
>
> Can I find the same info in the standard python doc or query python
> with
> a certain command to print out all existing codings?


Look under the heading "Standard Encodings":
http://docs.python.org/library/codecs.html

Note that both the page you found (which appears to be a copy of the
Python documentation) and the reference I provide say, "Neither the
list of aliases nor the list of languages is meant to be exhaustive".

I guess one reason for this is that different Python implementations
could choose to offer codecs for additional encodings.

"News123" <news123@free.fr> wrote in message
news:491779b1$0$19313$426a74cc@news.free.fr...
> Hi,
>
> I was googling quite some time before finding the answer to my question:
> 'what are the names for the encodings supported by python?'
>
> I found the answer at http://python.active-venture.com/lib/node127.html
>
>
> Now my question:
>
> Can I find the same info in the standard python doc or query python with
> a certain command to print out all existing codings?
>

The first hit from googling "siteython.org encodings":
http://www.python.org/doc/2.5.2/lib/...encodings.html

-Mark

Hi Philip,

Your answer touches exaclty one point, which I was slightly afraid of:
- The list is not exhaustive
- python versions might have implemented different codecs.

This is why I wondered whether there's any way of querying python for a
list of codecs it supports.

thanks again for your and the other answers


bye


N

Philip Semanchuk wrote:
>
> On Nov 9, 2008, at 7:00 PM, News123 wrote:
>
>> Hi,
>>
>> I was googling quite some time before finding the answer to my question:
>> 'what are the names for the encodings supported by python?'
>>
>> I found the answer at http://python.active-venture.com/lib/node127.html
>>
>>
>> Now my question:
>>
>> Can I find the same info in the standard python doc or query python with
>> a certain command to print out all existing codings?
>
>
> Look under the heading "Standard Encodings":
> http://docs.python.org/library/codecs.html
>
> Note that both the page you found (which appears to be a copy of the
> Python documentation) and the reference I provide say, "Neither the list
> of aliases nor the list of languages is meant to be exhaustive".
>
> I guess one reason for this is that different Python implementations
> could choose to offer codecs for additional encodings.

On Nov 11, 2008, at 9:10 AM, News123 wrote:

> Hi Philip,
>
> Your answer touches exaclty one point, which I was slightly afraid of:
> - The list is not exhaustive
> - python versions might have implemented different codecs.
>
> This is why I wondered whether there's any way of querying python
> for a
> list of codecs it supports.

Try this:
Python 2.5.1 (r251:54863, Nov 17 2007, 21:19:53)
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import encodings.aliases
>>>
>>> encodings.aliases.aliases


"aliases" in the encodings.aliases module is a dict mapping alias
names (the dict keys) to encodings (the dict values). Thus, this will
give you the list of supported encodings:
>>> set(encodings.aliases.aliases.values())


The encodings module isn't in the documentation (?!?); I found it when
looking through the Python source code. For that reason I can't say
more about how it works. You may want to experiment to see if
encodings added via codecs.register() show up in the
encodings.aliases.aliases dict.


Have fun
Philip



>
> Philip Semanchuk wrote:
>>
>> On Nov 9, 2008, at 7:00 PM, News123 wrote:
>>
>>> Hi,
>>>
>>> I was googling quite some time before finding the answer to my
>>> question:
>>> 'what are the names for the encodings supported by python?'
>>>
>>> I found the answer at http://python.active-venture.com/lib/node127.html
>>>
>>>
>>> Now my question:
>>>
>>> Can I find the same info in the standard python doc or query
>>> python with
>>> a certain command to print out all existing codings?
>>
>>
>> Look under the heading "Standard Encodings":
>> http://docs.python.org/library/codecs.html
>>
>> Note that both the page you found (which appears to be a copy of the
>> Python documentation) and the reference I provide say, "Neither the
>> list
>> of aliases nor the list of languages is meant to be exhaustive".
>>
>> I guess one reason for this is that different Python implementations
>> could choose to offer codecs for additional encodings.
> --
> http://mail.python.org/mailman/listinfo/python-list

Hi Philip,

Thanks for your answer:
The fact, that a module 'encodings' exists was new to me.


encodings.aliases.aliases has however one problem.
It helps to locate all encoding aliases, but it won't find entries for
which no aliases exist:

So I can find koi8_r and its aliases

[ [k,v] for k,v in encodings.aliases.aliases.iteritems() \
if v.find('koi')> -1 ]
> [['cskoi8r', 'koi8_r']]
However I wouuldn't find the greek code page 'cp737' as it exists
without an alias


[ [k,v] for k,v in encodings.aliases.aliases.iteritems() \
if v.find('737')> -1 ]
> []


What gives me a list of quite some encodings on my host is the shell command
ls /usr/lib/python2.5/encodings | sed -n 's/\.py$//p' | sort
(soma false hits, bit this is fine for me purposes)


I don't know if really all encodings are represented with a .py file and
if all encodigns have to be in this directory, but it's a start.

Using shell commands is not that pythonic:

I could try to rewrite this in python by
1.) determine from which directory encodings was imported and
then using the glob module to list all .py files located there.


thanks again and bye


N

Philip Semanchuk wrote:
>
> On Nov 11, 2008, at 9:10 AM, News123 wrote:
>
>> Hi Philip,
>>
>> Your answer touches exaclty one point, which I was slightly afraid of:
>> - The list is not exhaustive
>> - python versions might have implemented different codecs.
>>
>> This is why I wondered whether there's any way of querying python for a
>> list of codecs it supports.
>
> Try this:
> Python 2.5.1 (r251:54863, Nov 17 2007, 21:19:53)
> [GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import encodings.aliases
>>>>
>>>> encodings.aliases.aliases
>
>
> "aliases" in the encodings.aliases module is a dict mapping alias names
> (the dict keys) to encodings (the dict values). Thus, this will give you
> the list of supported encodings:
>>>> set(encodings.aliases.aliases.values())
>
>
> The encodings module isn't in the documentation (?!?); I found it when
> looking through the Python source code. For that reason I can't say more
> about how it works. You may want to experiment to see if encodings added
> via codecs.register() show up in the encodings.aliases.aliases dict.
>
>
> Have fun
> Philip
>
>
>
>>
>> Philip Semanchuk wrote:
>>>
>>> On Nov 9, 2008, at 7:00 PM, News123 wrote:
>>>
>>>> Hi,
>>>>
>>>> I was googling quite some time before finding the answer to my
>>>> question:
>>>> 'what are the names for the encodings supported by python?'
>>>>
>>>> I found the answer at http://python.active-venture.com/lib/node127.html
>>>>
>>>>
>>>> Now my question:
>>>>
>>>> Can I find the same info in the standard python doc or query python
>>>> with
>>>> a certain command to print out all existing codings?
>>>
>>>
>>> Look under the heading "Standard Encodings":
>>> http://docs.python.org/library/codecs.html
>>>
>>> Note that both the page you found (which appears to be a copy of the
>>> Python documentation) and the reference I provide say, "Neither the list
>>> of aliases nor the list of languages is meant to be exhaustive".
>>>
>>> I guess one reason for this is that different Python implementations
>>> could choose to offer codecs for additional encodings.
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>

On Nov 11, 2008, at 1:08 PM, News123 wrote:

> Hi Philip,
>
> Thanks for your answer:
> The fact, that a module 'encodings' exists was new to me.

We both learned something new today. =)


> encodings.aliases.aliases has however one problem.
> It helps to locate all encoding aliases, but it won't find entries for
> which no aliases exist:

Ooops, I hadn't thought about that.


> What gives me a list of quite some encodings on my host is the shell
> command
> ls /usr/lib/python2.5/encodings | sed -n 's/\.py$//p' | sort
> (soma false hits, bit this is fine for me purposes)
>
> I don't know if really all encodings are represented with a .py file
> and
> if all encodigns have to be in this directory, but it's a start.
>
>
> Using shell commands is not that pythonic:
>
> I could try to rewrite this in python by
> 1.) determine from which directory encodings was imported and
> then using the glob module to list all .py files located there.

Yes, I'd thought about this but I agree with you that it seems
unpythonic and fragile. Unfortunately I can't think of anything better
at this point.

Good luck
Philip


>
> Philip Semanchuk wrote:
>>
>> On Nov 11, 2008, at 9:10 AM, News123 wrote:
>>
>>> Hi Philip,
>>>
>>> Your answer touches exaclty one point, which I was slightly afraid
>>> of:
>>> - The list is not exhaustive
>>> - python versions might have implemented different codecs.
>>>
>>> This is why I wondered whether there's any way of querying python
>>> for a
>>> list of codecs it supports.
>>
>> Try this:
>> Python 2.5.1 (r251:54863, Nov 17 2007, 21:19:53)
>> [GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
>> Type "help", "copyright", "credits" or "license" for more
>> information.
>>>>> import encodings.aliases
>>>>>
>>>>> encodings.aliases.aliases
>>
>>
>> "aliases" in the encodings.aliases module is a dict mapping alias
>> names
>> (the dict keys) to encodings (the dict values). Thus, this will
>> give you
>> the list of supported encodings:
>>>>> set(encodings.aliases.aliases.values())
>>
>>
>> The encodings module isn't in the documentation (?!?); I found it
>> when
>> looking through the Python source code. For that reason I can't say
>> more
>> about how it works. You may want to experiment to see if encodings
>> added
>> via codecs.register() show up in the encodings.aliases.aliases dict.
>>
>>
>> Have fun
>> Philip
>>
>>
>>
>>>
>>> Philip Semanchuk wrote:
>>>>
>>>> On Nov 9, 2008, at 7:00 PM, News123 wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I was googling quite some time before finding the answer to my
>>>>> question:
>>>>> 'what are the names for the encodings supported by python?'
>>>>>
>>>>> I found the answer at http://python.active-venture.com/lib/node127.html
>>>>>
>>>>>
>>>>> Now my question:
>>>>>
>>>>> Can I find the same info in the standard python doc or query
>>>>> python
>>>>> with
>>>>> a certain command to print out all existing codings?
>>>>
>>>>
>>>> Look under the heading "Standard Encodings":
>>>> http://docs.python.org/library/codecs.html
>>>>
>>>> Note that both the page you found (which appears to be a copy of
>>>> the
>>>> Python documentation) and the reference I provide say, "Neither
>>>> the list
>>>> of aliases nor the list of languages is meant to be exhaustive".
>>>>
>>>> I guess one reason for this is that different Python
>>>> implementations
>>>> could choose to offer codecs for additional encodings.
>>> --
>>> http://mail.python.org/mailman/listinfo/python-list
>>
> --
> http://mail.python.org/mailman/listinfo/python-list

On Nov 10, 11:00 am, News123 <news...@free.fr> wrote:
> Hi,
>
> I was googling quite some time before finding the answer to my question:
> 'what are the names for the encodings supported by python?'
>
> I found the answer athttp://python.active-venture.com/lib/node127.html
>
> Now my question:
>
> Can I find the same info in the standard python doc or query python with
> a certain command to print out all existing codings?
>
> thanks in advance for your answer and bye
>
> N

You haven't explained why you think that you *need* a list of all
encodings that exist at a point in time. What are you going to do with
the list? Surely not use it to validate user input, one would hope.