Welcome Guest Search | Active Topics | Log In | Register

Currency Symbols and other ASCII Characters Options
Chris Bray
#1 Posted : Saturday, January 23, 2010 11:21:23 AM
Rank: Team VistaDB

Groups: Member, Team VistaDB

Joined: 8/13/2006
Posts: 393
Points: 657
Location: Seaford, East Sussex, UK
Hi Guys,

In case anyone else has an issue with text characters, I was having an issue with SQL Scripts related to Currency symbols where they were appearing in VistaDB as question marks for no good and sufficient reason that I could see. The queries worked, but the results were wrong. Jamie from VistaDB support gave me a pointer that helped to find the issue, and I thought that I would write it up in case it is useful to anyone else.

The SQL was written and tested in DataBuilder where it worked perfectly and then copied into a text file in Visual Studio 2008. It also had some of our own additional script code in it, but that can be ignored for the purposes of this discussion.

The functional contents of the file were:

Code:
UPDATE Currencies SET CurrencySymbol = '£' WHERE CurrencyCode IN ('CYP','EGP','FKP','GBP','GIP','SHP','SYP');
UPDATE Currencies SET CurrencySymbol = '€' WHERE CurrencyCode = 'EUR';
UPDATE Currencies SET CurrencySymbol = '¥' WHERE CurrencyCode = 'JPY';


This code works perfectly if copied back into DataBuilder and run, but not if it is loaded and run from the file.

The code to load the file was again trivial:

Code:
List<string> commandList = new List<string>();

// Load the command lines
using (StreamReader streamReader =
    new StreamReader(data.ScriptFileName))
{
    while ((currentLine = streamReader.ReadLine()) != null)
    {
        commandList.Add(currentLine);
    }
   
    streamReader.Close();
}


According to MSDN this should result in the file being read with the default encoding, but this was not happening. By simply explicitly setting the Default the problem went away:

Code:
// Load the command lines
using (StreamReader streamReader =
    new StreamReader(data.ScriptFileName, Encoding.Default))
{
    while ((currentLine = streamReader.ReadLine()) != null)
    {
        commandList.Add(currentLine);
    }
   
    streamReader.Close();
}


I still cannot understand why the initial version does not work correctly given that the currency symbols are well within the standard ASCII range i.e. Yen = 165, Pound = 163 Euro = 128, but at least it now works.

I hope that this is useful to someone.


Chris Bray
Vertical Software
http://www.verticalsoftware.co.uk
js_vistadb
#2 Posted : Monday, January 25, 2010 6:14:19 PM
Rank: VistaDB Staff

Groups: Administration, VistaDB Staff

Joined: 8/13/2006
Posts: 3,633
Points: 5,124
Location: Mount Dora, Florida
I think that Microsoft includes everything above Z in the ASCII chart to be extended ASCII. So any symbols will fail to load correctly.

It is a problem with the "default" constructor not using the Default behavior. We opened a ticket about this a while back and were told "as designed".

Incidentally this will also happen on Mono if you fail to specific a locale in your app since it doesn't know what locale to use.

The best advice - always be explicit about what you want a reader to read. If you think it might be UTF8 then load it that way. Since .Net is always Unicode under the covers you would think it would always understand Unicode by default, but it does not. Many of the built in .Net routines still fall back to unmanaged win32 defaults.

Jason Short

Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAFPro Theme Created by Jaben Cargman (Tiny Gecko)
Powered by YAF 1.9.3 | YAF © 2003-2009, Yet Another Forum.NET
This page was generated in 0.065 seconds.