XML Editor
Sign up for a WebBoard account Sign Up Keyword Search Search More Options... Options
search
Chat Rooms Chat Help Help News News Log in to WebBoard Log in Not Logged in
Home » Boards » Stylus Studio Developer Network » Stylus Studio Technical Forum » dodgy(non-ascii) characters causing confusion
Conferences Close Tree View
+ Stylus Studio Feature Requests (1192)
- Stylus Studio Technical Forum (14621)
-> + Disabled Watch and Variable Vi... (4)
-> + Accessing Filemaker DB via ODB... (2)
-> + About XML Graphs (2)
-> + Error when generate java bindi... (2)
-> + Formatting a number to always ... (2)
-> + Bug with graphical mapper (2)
-> + Back Map to XML does not work (2)
-> + Floating Instruction Graphics (2)
-> + Mapping XSD (4)
-> + Output has dashes (Hyphens) th... (7)
-> + Fwd: Error Message (2)
-> + Error Message (5)
-> + EPG XML Generation (2)
-> + Connection to DB2 UDB (8)
-> + Instructions In Mapper (6)
-> + using oracle's orcl:right-trim... (2)
-> + xsl fo, generate links in pdf... (3)
-> + Documentation (annotations) in... (4)
-> + Linking to RDBMS to Applicatio... (2)
-> + Stylus build 653c // Metainfor... (3)
-> + namesspacing external entities (6)
-> + Suppress namespace on every no... (5)
-> + xpath: how do I get what I wan... (6)
-> + Still can't remove SS!! (2)
-> + Convert to XML (3)
-> + EDI Error (10)
-> + (500) internal server error wh... (2)
-> + Space formatting with   (4)
-> + UDDI Browser and WS Call Compo... (4)
-> + cursor weirdness (2)
-> + Open XML files in IE, edit wit... (3)
-> + Generate XML from schema (2)
-> + Line-oriented region problem i... (4)
-> + Using local DB2 database Alias... (2)
-> + Small but annoying display bug... (3)
-> + Blank values in lookup lists (11)
-> + xsd problem in mapper (2)
-> + Deployment License (2)
-> + XML post and web service (2)
-> + Stylus just wont start? (2)
-> - Can'rt remove SS from Windows ... (1)
-> + SenseX stops working on second... (4)
-> - dodgy(non-ascii) characters ca... (8)
-> ->dodgy(non-ascii) characte...
-> ->dodgy(non-ascii) characte...
-> ->dodgy(non-ascii) characte...
-> ->dodgy(non-ascii) characte...
-> ->dodgy(non-ascii) characte...
-> ->dodgy(non-ascii) characte...
-> ->dodgy(non-ascii) characte...
-> + Optional values (ZZ1) causes e... (3)
-> + Have problems with displaying ... (9)
-> + Can Stylus Studio be used to m... (3)
-> + Erratic behaviour in EDIFACT a... (2)
-> + Year Calculation (2)
-> + How to ouput a real XLS (3)
-> + EDI 856 to XML - java.io.IOExc... (3)
-> + Problem transform EDI to XML (... (2)
-> + SS becomes unusable... (6)
-> + SS Enterprise 735f under WinXP... (7)
-> + Call Web Service, Nothing retu... (12)
-> + Error while debugging xslt wit... (2)
-> + DESADV D96A Schema vs XML inco... (4)
-> + Das Programm mit mysql ? (4)
-> + RegEx Search and Replace (6)
-> + SequeLink Server - Do I need a... (6)
-> + faultString: (401)Unauthorized... (7)
-> + Previewing XSL with javascript... (2)
-> + removing formating from XML (6)
-> + removing formating from XML (2)
-> + Custom EDI to XML conversion (2)
-> + Getting "Type not found" error... (4)
-> - Converting file and customize ... (1)
-> + formatting problem with xml ex... (2)
-> + Using DTD Entities and XSD (7)
-> + XML Schema (XSD), Adding comme... (2)
-> + DTD schema generation problem (2)
-> + How do i generate a schema fro... (4)
-> + Incomplete request message gen... (2)
-> + Installation issue for Stylus ... (2)
-> + DTD "ANY" isn't honored by val... (2)
-> + Replacing carriage returns wit... (3)
-> + WS with SOAP - problem with pa... (5)
-> + problem converting CSV to XML (2)
-> + validating xml with multiple s... (4)
-> + SOAP problems ! (2)
-> + mapping csv columns to schema (2)
-> - Access DB to XML doc using an ... (1)
-> - DataDirect SequelLink w/MS Acc... (1)
-> + Problem with DB to XML Source ... (3)
-> + problems to work from the comm... (3)
-> + Stylus Studio Enterprise WScal... (4)
-> + Erratic behaviour in EDIFACT a... (14)
-> + XML Pipeline Question (7)
-> + Stylus Studio reports invalid ... (4)
-> - Is there a way to generate a G... (1)
-> + Stylus produced XSLT not trans... (2)
-> + XML Editor spell check - bug w... (4)
-> + API unlocks OK, but fails in n... (4)
-> + EDI to XML conversion for UNOC... (11)
-> + Stylus crashed with recurence ... (2)
-> + terrible perfomance using the ... (4)
-> + XSD to RDBMS, e.g. SQL Server,... (3)
-> + relational database (7)
-> + Stylus Studio 6 wrongfully det... (6)
-> + Having trouble with schema val... (5)
-> + Stylus Studio 2007 Home Editio... (2)
-- Previous [1681-1700] [1701-1720] [1721-1740] Next
+ Website Feedback (249)
+ XSLT Help and Discussion (7625)
+ XQuery Help and Discussion (2017)
+ Stylus Studio FAQs (159)
+ Stylus Studio Code Samples & Utilities (364)
+ Stylus Studio Announcements (113)
Topic  
Postnext
Richard PottsSubject: dodgy(non-ascii) characters causing confusion
Author: Richard Potts
Date: 14 Feb 2007 11:21 AM
Originally Posted: 14 Feb 2007 11:19 AM
SS Version: Enterprise 2006 Release 2 591d

I've written a XSLT to convert from XML to another form of XML.

Extract from source document:

<did id="0xF18C" name="ecu_sn">
<hex>FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF</hex>
<txt>
<element> <name>value</name> <value>˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙</value>
</element>
</txt>
</did>


Extract from resulting XMLdocument:

<SerialNumber Name="ecu_sn">
<Value>&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;</Value>
</SerialNumber>


(The &#65533;&#65533;&#65533; chars actually appear as "?" within a black diamond. within SS)

What I don't understand is that when viewing the source document in SS it 'renders' ok but the resultant xml doesn't. The problem has something to do with the 'dodgy' characters '˙˙˙˙˙˙˙˙˙˙˙˙˙˙'. But why is my resulting XML in error? (When all I'm doing is 'echoing' the dodgy characters to the output xml?)

For reference, Extract from my XSLT:

<xsl:variable name="SerialNumberValue">
<xsl:value-of select="txt/element/value"/>
</xsl:variable>
<xsl:choose>
<xsl:when test="$SerialNumberValue != ''">
<SerialNumber>
<xsl:attribute name="Name">
<xsl:value-of select="@name"/>
</xsl:attribute>
<Value>
<xsl:value-of select="$SerialNumberValue"></xsl:value-of>
</Value>
</SerialNumber>
</xsl:when>
<xsl:otherwise>
<SerialNumber>
<xsl:attribute name="Name">
<xsl:value-of select="@name"/>
</xsl:attribute>
<Value>
<xsl:text>?</xsl:text>
</Value>
</SerialNumber>
<Error><xsl:value-of select="concat('Module Serial Number is blank: ',../../../@name,' ',@name)"></xsl:value-of></Error>
</xsl:otherwise>
</xsl:choose>
Postnext
(Deleted User) Subject: dodgy(non-ascii) characters causing confusion
Author: (Deleted User)
Date: 14 Feb 2007 12:12 PM
Hi Richard,
could you attach a screenshot of Stylus? It's not clear to me where the characters are displayed properly and where they aren't.

Thanks,
Alberto
Postnext
Richard PottsSubject: dodgy(non-ascii) characters causing confusion
Author: Richard Potts
Date: 14 Feb 2007 12:23 PM
Find screen shots attached.

Note: the XSLT I developend in SS but apply it in my VBA application.
I've now tracked problem down to someting to do with 'encoding' because if I open up my resultant file in notepad and then "save as.." it says 'ANSI' and if I save it as "UTF-8" IE will open it ok.

Where as if I do the same with my souce document it says already 'UTF-8'

So where do I tell my application to save the resultant XML file to be UTF-8? in the stylesheet or somewhere in VBA or both?
Postnext
Richard PottsSubject: dodgy(non-ascii) characters causing confusion
Author: Richard Potts
Date: 14 Feb 2007 12:23 PM
Find screen shots attached.

Note: the XSLT I developend in SS but apply it in my VBA application.
I've now tracked problem down to someting to do with 'encoding' because if I open up my resultant file in notepad and then "save as.." it says 'ANSI' and if I save it as "UTF-8" IE will open it ok.

Where as if I do the same with my souce document it says already 'UTF-8'

So where do I tell my application to save the resultant XML file to be UTF-8? in the stylesheet or somewhere in VBA or both?

Thanks in advance.


Unknownscreenshots.doc
Postnext
(Deleted User) Subject: dodgy(non-ascii) characters causing confusion
Author: (Deleted User)
Date: 14 Feb 2007 01:02 PM
Richard,
it depend how your VBA application saves the file... if it relies on the MSXSL processor to actually write the file, it would be enough to add a <xsl:output encoding="utf-8"/> instruction right after the <xsl:stylesheet> node.
If your application is instead getting the result as a VBA string, you should investigate if the VBA file allows writing data in Unicode, instead of using the current locale.

Alberto
Postnext
Richard PottsSubject: dodgy(non-ascii) characters causing confusion
Author: Richard Potts
Date: 15 Feb 2007 06:48 AM
Thanks for the pointer Alberto.

In my VBA code I was creating the resultant file using "Writeline" i.e.
'
'Output to file
'
ExtractFile.WriteLine xslProc.output


Further Investigation on the web, for similar problems - found that ADO 'streams' seemed to be the answer: I've now changed it to:

'
' Open stream and set output code to UTF-8
'
Str.Charset = "UTF-8"
Str.Open
xslProc.output = Str

'
' Apply Transform
'
xslProc.transform

'
' Save stream to file
'
Str.SaveToFile sExtractFileName


So hope that helps someone else.

Not sure if it is possible or not… but as a thought, is there a potential enhancement to SS here? – e.g. to have a 'properties' for each file loaded into the XML editor so you can see what 'encoding' it was created with. Also is it possible to high-light the encoding issue to the user when you get the "? in diamonds" displayed for a xml file - How about maybe a SS feature 'convert this file from one encoding format to another encoding format' ???
Postnext
Tony LavinioSubject: dodgy(non-ascii) characters causing confusion
Author: Tony Lavinio
Date: 15 Feb 2007 09:40 AM
The XML standard says that if the encoding isn't in the first
line of the file, then it MUST be only one of a handful. The
rules are very specific, and anything else should be considered
broken and rejected by the parser.

See http://www.w3.org/TR/2006/REC-xml-20060816/#sec-guessing

The problem is that if the actual encoding of the file differs
from the stated encoding, then it is not always possible to determine
unambiguously what the encoding should have been. It is possible to
have a file that could be read successfully using several different
encodings and yielding several different results.

We do the best we can, but it is not deterministically possible to
tell in all cases, so the safest course is for Stylus Studio to
complain.
Posttop
Tony LavinioSubject: dodgy(non-ascii) characters causing confusion
Author: Tony Lavinio
Date: 15 Feb 2007 09:42 AM
... and there is a way inside of Stylus Studio to change the encoding,
but it assumes that the file in the editor is displayed properly.

Really, to fix a broken encoding you'd need a non-unicode-aware tool,
or more ideally you would get the source of the broken XML to not write
invalid XML.
   
Download A Free Trial of Stylus Studio 6 XML Professional Edition Today! Powered by Stylus Studio, the world's leading XML IDE for XML, XSLT, XQuery, XML Schema, DTD, XPath, WSDL, XHTML, SQL/XML, and XML Mapping!  
go

Log In Options

Site Map | Privacy Policy | Terms of Use | Trademarks
Stylus Scoop XML Newsletter:
W3C Member
Stylus Studio® and DataDirect XQuery ™are from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2016 All Rights Reserved.