Swazoo encoding

Hi there,

I'm able to use swazoo with proper encoding for all content that comes
an goes by http. My encoding problem is with the strings generated in
the dolphin environment.
Those which has diacritics wont render properly and I need those
badly.
Any solution known?
Sebastian

On Aug 27, 10:31 pm, Sebastian <ssas...@seaswork.com.ar> wrote:
> Hi there,
>
> I'm able to use swazoo with proper encoding for all content that comes
> an goes by http. My encoding problem is with the strings generated in
> the dolphin environment.
> Those which has diacritics wont render properly and I need those
> badly.
> Any solution known?
> Sebastian

I assume that problem is that strings that you have captured through
dolphin edit box (or constants in code entered through method editor),
show up garbled on the web browser once they arrive there somehow?

Dolphin internally uses some windows code page, and you could write
translator that translates a string from windows encoding to utf8
encoding. Good thing is that you can hold utf8 strings in standard
dolphin strings, and as long you do not need to check its length or
stream over it one unicode character by one unicode, most things
working ok, as long as you keep in mind that some unicode characters
get encoded by more than one regular character.

In your edit boxes you can set TypeConverter property, and most of
your strings would get converted to and from utf8 automagically.

Alternatively, if your data input needs can be solved by Scintilla
text boxes, with some poking aroound scintilla can be set to unicode
mode, where you suply it directly utf8 encoded string, and you also
get the result utf8 encoded.


Hope that helps,

Davorin

Some strings came from code other is read from text files.
But no matter where it came from it reach HTTPResponse>>entity: as a
dolphin string which converted to a byte array makes a ú to be 250 and
reach the UA as a strange "?" character.
It probably need to be encoded before that point.
cheers,
Sebastian

You will have to use utf-8 internally or make some conversion before
generating HTML.

Or you can change HTML generating code so that it will include <meta
http-equiv="Content-Type" content="text/html; charset=windows-1252"/> in
the page header. But even with this workaround you will still need a way
to convert to utf-8 for Ajax requests, since XMLHttpRequest in IE works
only with utf-8.

Best regards, David


Sebastian wrote:
> Some strings came from code other is read from text files.
> But no matter where it came from it reach HTTPResponse>>entity: as a
> dolphin string which converted to a byte array makes a ú to be 250 and
> reach the UA as a strange "?" character.
> It probably need to be encoded before that point.
> cheers,
> Sebastian

On Aug 28, 10:33*am, David Gorisek <da...@remove-this.gorisek.com>
wrote:
> You will have to use utf-8 internally or make some conversion before
> generating HTML.
>
> Or you can change HTML generating code so that it will include *<meta
> http-equiv="Content-Type" content="text/html; charset=windows-1252"/> in
> the page header. But even with this workaround you will still need a way
> to convert to utf-8 for Ajax requests, since XMLHttpRequest in IE works
> only with utf-8.
>
> Best regards, David
>
Hi David,
after seeing this:
http://www.nabble.com/VisualWorks-%2...html#a11778503

I'm trying the approach of letting header content in utf-8 and make
the internal conversion like this:

HTTPResponse>>entity: anEntity
entity := anEntity isString
ifTrue:[anEntity asUTF8 asByteArray]
ifFalse:[anEntity asByteArray]

it wont show ulgy chars but any diacritic simply disappears.
Can someone confirm if Swazoo is expecting to encode an unicode string
instead of a common one?

I don't see why that wont work.
thanks for helping,
Sebastian

David,

I've double checked

'ñandú' asUTF8 is answering ' and ' instead of 'ñandú' (this one
decodes properly to 'ñandú' when sent #webUTF8Decoded).

In the other hand:

' and ' webUTF8Decoded

is answering ' and '

so it clearly misses something.
I'm most wiling to fix that. Any pointer?
thanks,
Sebastian

Sebastian,

I see you are using the methods added by the Dialect Abstraction Layer
which are used by my web server and are required by the WikiDoc and STS
server packages.

These methods clearly wont work with your example simply because we have
only implemented complete support for some east European languages plus
English and German.

The other thing with your code is that there are two methods #asUTF8 and
#asUTF8String where the first one also adds a Unicode file marker (4
bytes) before the resulting string (see the method comment). So the code
to check the conversion should be something like this:

'string' asUTF8String webUTF8Decoded

And there is even more that you need to know before using it. The
conversion needs to know which code page is being used. And this is why
it sends the method Process>>#webRequestCodePage

If you want to use this for conversion to UTF8, then you will have to
add the additional characters that you need in Spanish to the dictionary
which is mapping special characters to Unicode. See method
WebTranslationService>>#defineIsoCodesMapCPWindows 1252

If you do this, then just upload your changes to the public repository
and I will integrate them into WikiDoc, or you can even add support for
Spanish language.

But all this code is of course not part of Seaside so you may have to
look for another solution which is meant to be used with Seaside...

Best regards,

David



Sebastian wrote:
> David,
>
> I've double checked
>
> 'ñandú' asUTF8 is answering ' and ' instead of 'ñandú' (this one
> decodes properly to 'ñandú' when sent #webUTF8Decoded).
>
> In the other hand:
>
> ' and ' webUTF8Decoded
>
> is answering ' and '
>
> so it clearly misses something.
> I'm most wiling to fix that. Any pointer?
> thanks,
> Sebastian

Done. Thank you so much David.
Do I need to request an user to post in the public repo?
Sebastian
PD: My version of your web server didn't install #asUTF8String (only
#asUTF8) but I've removed the header and all went well. Also I've
expanded 1252 table to spanish and portuguese diacritic chars. Now all
major browsers are with utf8 in render and ajax and the app wont
notice.