XML Editor
Sign up for a WebBoard account Sign Up Keyword Search Search More Options... Options
search
Chat Rooms Chat Help Help News News Log in to WebBoard Log in Not Logged in
Home » Boards » Stylus Studio Developer Network » Stylus Studio Technical Forum » dodgy(non-ascii) characters causing confusion
Conferences Close Tree View
+ Stylus Studio Feature Requests (1192)
- Stylus Studio Technical Forum (14621)
-> - More flexible generated Java c... (1)
-> + Generic Schemas for representi... (2)
-> + Stylus Studio Professional rep... (2)
-> + XML XML Help (2)
-> + xsd help (2)
-> + XML data element friendly name... (2)
-> + HTML Tidy adapter is missing? (3)
-> + New Guy Here... (2)
-> + How to combine two similar fil... (2)
-> + Berkeley XML DB (4)
-> + What is the context node in th... (3)
-> + Multi element List (4)
-> + Cannot preview XSL transformat... (9)
-> + Why is this variable not showi... (7)
-> + Customising Schema Prefix (3)
-> + Please help!! XML AND XSL Tra... (6)
-> + XML to Excel in Home Edition (2)
-> + how to copy elements from an x... (2)
-> + Setting Saxon parameters in pi... (2)
-> + Trying to move SS license (3)
-> + Does SS PRO Support CSS (2)
-> + Selected values are not showin... (3)
-> + Copy Xpath To Clipboard Option... (2)
-> + Upgrading to SS 2006 3 - Activ... (3)
-> + XSL Efficiency tips (4)
-> - Report page Size and table hea... (1)
-> + Endless loop with xquery (8)
-> - Xpress XML (1)
-> + Extracting XML data into SQL s... (2)
-> + Crashes when I save (2)
-> + XSL cannot save: I/O error...b... (8)
-> + Diagram view has disappeared f... (5)
-> + Convert from http://www.w3.org... (2)
-> + Round-trip Converting; CSV-XML... (2)
-> + XSL error function call PHP (8)
-> + WYSIWYG XSLT Designer (5)
-> + generate xhtml+css from xml re... (2)
-> + generate xhtml+css from xml re... (3)
-> + generate xhtml+css from xml re... (2)
-> - Help using xsd xs:redefine nee... (1)
-> + Data loss opening External sch... (2)
-> + Install Apache FOP Plugin with... (8)
-> + Generating a Publisher Canvas ... (2)
-> + Stylus Studio comment block no... (3)
-> + Report Designer Screen Repaint... (5)
-> + validating on values (5)
-> + reuse of xslt (2)
-> + The evaluation period for the ... (2)
-> + export to documentation (3)
-> + Validating XML files using sev... (2)
-> + Missing 'Command' Tab in Custo... (3)
-> + Do I have to download new vers... (6)
-> + Can SS validate tags used in a... (5)
-> + validating schemas (2)
-> + Off-Topic Question - Looking f... (4)
-> + cannot see anything in mapper ... (2)
-> + Validating XSD schema sets (7)
-> + Standard Validation Engines (3)
-> + Style Sheet amendment to inclu... (2)
-> + Documentation - prevent import... (3)
-> + Debugging breakpoints with Sax... (17)
-> + Conversion (3)
-> + New and I am sure this is an e... (7)
-> + Saxon 8 with multiple xml inpu... (3)
-> + Compilation error (5)
-> + webservice call failing - pls ... (3)
-> + C# from Java Binding Classes (3)
-> + Java binding class generation (5)
-> + Validation of deeply-nested sc... (5)
-> + How to make Drag and drop in X... (2)
-> + Malformed XPATH expression par... (5)
-> + Toolbars and missing options (4)
-> + Converting Text File (2)
-> + Java Heap Issue (2)
-> + Formatting SOAP messages (6)
-> + Attribute Entity Expansion on ... (2)
-> + Text File to xml conversion wi... (9)
-> + Different XPath results when u... (7)
-> + Where is my XSLT mapper module (3)
-> + Where is my XSLT WYSIWYG Edito... (2)
-> + Regarding Stylus support of IA... (2)
-> + Debugging Problem with Enterpr... (5)
-> + various report designer prob;e... (12)
-> - will it EVER get fixed ?? Stru... (1)
-> - Migrate from wysiwyg xslt edit... (1)
-> + Entity Resolver in UI? (4)
-> + Visual Studio and Stylus Studi... (2)
-> + how to get rid of parameter in... (8)
-> + XQuery file (2)
-> + 505 error calling a WS from SS... (2)
-> + Configure Subversion with Styl... (2)
-> + XSLT Mapping of xsi:nil and xs... (2)
-> + Printing values in a table or ... (2)
-> + query on using stylus API for ... (12)
-> + WSDL retruns no response (2)
-> + SS XSL Processor Bug: Count() (6)
-> + Stylus Studio crashes after re... (2)
-> + Custom Converter in IBM JVM is... (10)
-> + Malformed XPATH epressions on ... (4)
-> - XML Report table borders when ... (1)
-- Previous [1501-1520] [1521-1540] [1541-1560] Next
+ Website Feedback (249)
+ XSLT Help and Discussion (7625)
+ XQuery Help and Discussion (2017)
+ Stylus Studio FAQs (159)
+ Stylus Studio Code Samples & Utilities (364)
+ Stylus Studio Announcements (113)
Topic  
Postnext
Richard PottsSubject: dodgy(non-ascii) characters causing confusion
Author: Richard Potts
Date: 14 Feb 2007 11:21 AM
Originally Posted: 14 Feb 2007 11:19 AM
SS Version: Enterprise 2006 Release 2 591d

I've written a XSLT to convert from XML to another form of XML.

Extract from source document:

<did id="0xF18C" name="ecu_sn">
<hex>FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF</hex>
<txt>
<element> <name>value</name> <value>˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙</value>
</element>
</txt>
</did>


Extract from resulting XMLdocument:

<SerialNumber Name="ecu_sn">
<Value>&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;</Value>
</SerialNumber>


(The &#65533;&#65533;&#65533; chars actually appear as "?" within a black diamond. within SS)

What I don't understand is that when viewing the source document in SS it 'renders' ok but the resultant xml doesn't. The problem has something to do with the 'dodgy' characters '˙˙˙˙˙˙˙˙˙˙˙˙˙˙'. But why is my resulting XML in error? (When all I'm doing is 'echoing' the dodgy characters to the output xml?)

For reference, Extract from my XSLT:

<xsl:variable name="SerialNumberValue">
<xsl:value-of select="txt/element/value"/>
</xsl:variable>
<xsl:choose>
<xsl:when test="$SerialNumberValue != ''">
<SerialNumber>
<xsl:attribute name="Name">
<xsl:value-of select="@name"/>
</xsl:attribute>
<Value>
<xsl:value-of select="$SerialNumberValue"></xsl:value-of>
</Value>
</SerialNumber>
</xsl:when>
<xsl:otherwise>
<SerialNumber>
<xsl:attribute name="Name">
<xsl:value-of select="@name"/>
</xsl:attribute>
<Value>
<xsl:text>?</xsl:text>
</Value>
</SerialNumber>
<Error><xsl:value-of select="concat('Module Serial Number is blank: ',../../../@name,' ',@name)"></xsl:value-of></Error>
</xsl:otherwise>
</xsl:choose>
Postnext
(Deleted User) Subject: dodgy(non-ascii) characters causing confusion
Author: (Deleted User)
Date: 14 Feb 2007 12:12 PM
Hi Richard,
could you attach a screenshot of Stylus? It's not clear to me where the characters are displayed properly and where they aren't.

Thanks,
Alberto
Postnext
Richard PottsSubject: dodgy(non-ascii) characters causing confusion
Author: Richard Potts
Date: 14 Feb 2007 12:23 PM
Find screen shots attached.

Note: the XSLT I developend in SS but apply it in my VBA application.
I've now tracked problem down to someting to do with 'encoding' because if I open up my resultant file in notepad and then "save as.." it says 'ANSI' and if I save it as "UTF-8" IE will open it ok.

Where as if I do the same with my souce document it says already 'UTF-8'

So where do I tell my application to save the resultant XML file to be UTF-8? in the stylesheet or somewhere in VBA or both?
Postnext
Richard PottsSubject: dodgy(non-ascii) characters causing confusion
Author: Richard Potts
Date: 14 Feb 2007 12:23 PM
Find screen shots attached.

Note: the XSLT I developend in SS but apply it in my VBA application.
I've now tracked problem down to someting to do with 'encoding' because if I open up my resultant file in notepad and then "save as.." it says 'ANSI' and if I save it as "UTF-8" IE will open it ok.

Where as if I do the same with my souce document it says already 'UTF-8'

So where do I tell my application to save the resultant XML file to be UTF-8? in the stylesheet or somewhere in VBA or both?

Thanks in advance.


Unknownscreenshots.doc
Postnext
(Deleted User) Subject: dodgy(non-ascii) characters causing confusion
Author: (Deleted User)
Date: 14 Feb 2007 01:02 PM
Richard,
it depend how your VBA application saves the file... if it relies on the MSXSL processor to actually write the file, it would be enough to add a <xsl:output encoding="utf-8"/> instruction right after the <xsl:stylesheet> node.
If your application is instead getting the result as a VBA string, you should investigate if the VBA file allows writing data in Unicode, instead of using the current locale.

Alberto
Postnext
Richard PottsSubject: dodgy(non-ascii) characters causing confusion
Author: Richard Potts
Date: 15 Feb 2007 06:48 AM
Thanks for the pointer Alberto.

In my VBA code I was creating the resultant file using "Writeline" i.e.
'
'Output to file
'
ExtractFile.WriteLine xslProc.output


Further Investigation on the web, for similar problems - found that ADO 'streams' seemed to be the answer: I've now changed it to:

'
' Open stream and set output code to UTF-8
'
Str.Charset = "UTF-8"
Str.Open
xslProc.output = Str

'
' Apply Transform
'
xslProc.transform

'
' Save stream to file
'
Str.SaveToFile sExtractFileName


So hope that helps someone else.

Not sure if it is possible or not… but as a thought, is there a potential enhancement to SS here? – e.g. to have a 'properties' for each file loaded into the XML editor so you can see what 'encoding' it was created with. Also is it possible to high-light the encoding issue to the user when you get the "? in diamonds" displayed for a xml file - How about maybe a SS feature 'convert this file from one encoding format to another encoding format' ???
Postnext
Tony LavinioSubject: dodgy(non-ascii) characters causing confusion
Author: Tony Lavinio
Date: 15 Feb 2007 09:40 AM
The XML standard says that if the encoding isn't in the first
line of the file, then it MUST be only one of a handful. The
rules are very specific, and anything else should be considered
broken and rejected by the parser.

See http://www.w3.org/TR/2006/REC-xml-20060816/#sec-guessing

The problem is that if the actual encoding of the file differs
from the stated encoding, then it is not always possible to determine
unambiguously what the encoding should have been. It is possible to
have a file that could be read successfully using several different
encodings and yielding several different results.

We do the best we can, but it is not deterministically possible to
tell in all cases, so the safest course is for Stylus Studio to
complain.
Posttop
Tony LavinioSubject: dodgy(non-ascii) characters causing confusion
Author: Tony Lavinio
Date: 15 Feb 2007 09:42 AM
... and there is a way inside of Stylus Studio to change the encoding,
but it assumes that the file in the editor is displayed properly.

Really, to fix a broken encoding you'd need a non-unicode-aware tool,
or more ideally you would get the source of the broken XML to not write
invalid XML.
   
Download A Free Trial of Stylus Studio 6 XML Professional Edition Today! Powered by Stylus Studio, the world's leading XML IDE for XML, XSLT, XQuery, XML Schema, DTD, XPath, WSDL, XHTML, SQL/XML, and XML Mapping!  
go

Log In Options

Site Map | Privacy Policy | Terms of Use | Trademarks
Stylus Scoop XML Newsletter:
W3C Member
Stylus Studio® and DataDirect XQuery ™are from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2016 All Rights Reserved.