XML Editor
Sign up for a WebBoard account Sign Up Keyword Search Search More Options... Options
search
Chat Rooms Chat Help Help News News Log in to WebBoard Log in Not Logged in
Home » Boards » Stylus Studio Developer Network » Stylus Studio Technical Forum » dodgy(non-ascii) characters causing confusion
Conferences Close Tree View
+ Stylus Studio Feature Requests (1192)
- Stylus Studio Technical Forum (14621)
-> + XML Publisher Generation of XS... (3)
-> + SS Error: Segment UNB: Segment... (6)
-> + SS Enterprise 2007 R2 does not... (2)
-> + Generating XSD from XML - glob... (3)
-> + SS Enterprise 2007 R2 does not... (2)
-> + SS dies on xsl files loads (7)
-> + EDIFACT UNA section ? (3)
-> + SS closes when I attempt to sa... (2)
-> + Support for SPEC 2000 & genera... (6)
-> + Uninstall failure (10)
-> + WSDL to JAVA code (2)
-> + Can't Connect to DB - SQL 2005... (3)
-> + Java editor (3)
-> + Trouble getting my xsl scenari... (6)
-> + Activation Key Not Compatible.... (2)
-> + Stylus Studio 2007 R2 and Regi... (3)
-> - XSD file displayed in Excel an... (1)
-> + What Is This vmm File? (5)
-> + Another in Release 2 (3)
-> + Error on Release 2 (3)
-> + Query DB and convert output to... (11)
-> + import static HTML into XML Pu... (2)
-> + saxon 8.9 include path resolut... (10)
-> + Can't drag field delimiter (2)
-> + Enterprise Edition Relase 2 up... (4)
-> + Upgrade to Enterprise Release ... (2)
-> + HTML to XSLT wizard missing in... (7)
-> + sqlxml.dll error after update ... (4)
-> + Printing the tree or the grid ... (2)
-> + Sorting problem with xsl[decim... (2)
-> + 2007 R2 Saxon: missing Convert... (4)
-> + EDIFACT D96A INVRPT into a SQL... (2)
-> + any addon Feature or tool that... (2)
-> + Catalog Assistance - (3)
-> + WYSIWYG does not show (3)
-> + HL7 support in Stylus (2)
-> + Xerces parser missing from Sty... (4)
-> - automatic DTD verification bef... (1)
-> + Try this (2)
-> - 2007 R2 java code generation -... (1)
-> + 2007 R2 Java built-in XSLT (4)
-> + 2007 R2 saxon 8.9 debug proble... (2)
-> + Stylus Studio Schema Validatio... (3)
-> + Bug: losing items when editing... (2)
-> + 2007 R2 Java built-in XSLT (2)
-> + XML->EDI: inconsistent treatme... (13)
-> + 2007 R2 global java classpath (2)
-> + New Web Service Call Failing (6)
-> + Grid view for nested tables (2)
-> - Revisit Old Problem from last ... (1)
-> + Install Question (3)
-> + hex value problem (5)
-> + WSDL Overview Diagram? (2)
-> + Outputing HTML from an XSLT 2.... (2)
-> + Stylus Studio Schema Publicati... (2)
-> + Re-usable XML Schema (3)
-> + .net xslt extension objects (6)
-> + InstallShield problem (2)
-> + Changing default XSLT processo... (2)
-> - SS incorrectly handles paramet... (1)
-> + Can't add file to project (7)
-> + Stylus invoking when i login i... (2)
-> + How to utilize Saxon (custom) ... (3)
-> + xsl:decimal-format (5)
-> + Hidden Costs of SS? (2)
-> + Eclipse Plugin available? (4)
-> + Having trouble with xpath quer... (2)
-> + Specifying processor to use wi... (3)
-> + MySQL or Generic JDBC Data Sup... (8)
-> + JAXB java generation doesn't s... (3)
-> - Eclipse (1)
-> + XSTL Debugger not working for ... (6)
-> + SS w/ Visual Studio and VSS (2)
-> + SS crashes on load (2)
-> + Can't find file ... then does,... (2)
-> + Diff folders not working (4)
-> + last node in preceding-sibling (3)
-> + SQLXML and Subquery (2)
-> + Xquery - No ReleationalDB opti... (3)
-> + Is a default adapter - not Tid... (8)
-> + Need actual (18)
-> + include path for external enti... (2)
-> + xpath query in SQL server (4)
-> + XPATH Query that works in 2006... (4)
-> + Need help with XML Assignment (3)
-> + Java Compiler in Windows Vista... (6)
-> + Sense:X Only Shows One Enumera... (3)
-> + adding tables and relationship... (6)
-> + preview automatically starts a... (12)
-> + French date in excel 97 using ... (2)
-> + How to maintain large XSL (2)
-> + French date in excel 97 using ... (2)
-> + How to pass EDIFACT as paramet... (3)
-> + C# Generation from XSD (5)
-> + Oracle RDBMS <-> .XSD (2)
-> + Support for Berkeley DB XML 2.... (2)
-> + conrext lost with xslt 2.0 for... (4)
-> + BizTalk schema problem (4)
-> + MSXML 6 properties resolveExte... (3)
-> + Deployment adapters and lookup... (2)
-- Previous [1621-1640] [1641-1660] [1661-1680] Next
+ Website Feedback (249)
+ XSLT Help and Discussion (7625)
+ XQuery Help and Discussion (2017)
+ Stylus Studio FAQs (159)
+ Stylus Studio Code Samples & Utilities (364)
+ Stylus Studio Announcements (113)
Topic  
Postnext
Richard PottsSubject: dodgy(non-ascii) characters causing confusion
Author: Richard Potts
Date: 14 Feb 2007 11:21 AM
Originally Posted: 14 Feb 2007 11:19 AM
SS Version: Enterprise 2006 Release 2 591d

I've written a XSLT to convert from XML to another form of XML.

Extract from source document:

<did id="0xF18C" name="ecu_sn">
<hex>FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF</hex>
<txt>
<element> <name>value</name> <value>˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙</value>
</element>
</txt>
</did>


Extract from resulting XMLdocument:

<SerialNumber Name="ecu_sn">
<Value>&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;</Value>
</SerialNumber>


(The &#65533;&#65533;&#65533; chars actually appear as "?" within a black diamond. within SS)

What I don't understand is that when viewing the source document in SS it 'renders' ok but the resultant xml doesn't. The problem has something to do with the 'dodgy' characters '˙˙˙˙˙˙˙˙˙˙˙˙˙˙'. But why is my resulting XML in error? (When all I'm doing is 'echoing' the dodgy characters to the output xml?)

For reference, Extract from my XSLT:

<xsl:variable name="SerialNumberValue">
<xsl:value-of select="txt/element/value"/>
</xsl:variable>
<xsl:choose>
<xsl:when test="$SerialNumberValue != ''">
<SerialNumber>
<xsl:attribute name="Name">
<xsl:value-of select="@name"/>
</xsl:attribute>
<Value>
<xsl:value-of select="$SerialNumberValue"></xsl:value-of>
</Value>
</SerialNumber>
</xsl:when>
<xsl:otherwise>
<SerialNumber>
<xsl:attribute name="Name">
<xsl:value-of select="@name"/>
</xsl:attribute>
<Value>
<xsl:text>?</xsl:text>
</Value>
</SerialNumber>
<Error><xsl:value-of select="concat('Module Serial Number is blank: ',../../../@name,' ',@name)"></xsl:value-of></Error>
</xsl:otherwise>
</xsl:choose>
Postnext
(Deleted User) Subject: dodgy(non-ascii) characters causing confusion
Author: (Deleted User)
Date: 14 Feb 2007 12:12 PM
Hi Richard,
could you attach a screenshot of Stylus? It's not clear to me where the characters are displayed properly and where they aren't.

Thanks,
Alberto
Postnext
Richard PottsSubject: dodgy(non-ascii) characters causing confusion
Author: Richard Potts
Date: 14 Feb 2007 12:23 PM
Find screen shots attached.

Note: the XSLT I developend in SS but apply it in my VBA application.
I've now tracked problem down to someting to do with 'encoding' because if I open up my resultant file in notepad and then "save as.." it says 'ANSI' and if I save it as "UTF-8" IE will open it ok.

Where as if I do the same with my souce document it says already 'UTF-8'

So where do I tell my application to save the resultant XML file to be UTF-8? in the stylesheet or somewhere in VBA or both?
Postnext
Richard PottsSubject: dodgy(non-ascii) characters causing confusion
Author: Richard Potts
Date: 14 Feb 2007 12:23 PM
Find screen shots attached.

Note: the XSLT I developend in SS but apply it in my VBA application.
I've now tracked problem down to someting to do with 'encoding' because if I open up my resultant file in notepad and then "save as.." it says 'ANSI' and if I save it as "UTF-8" IE will open it ok.

Where as if I do the same with my souce document it says already 'UTF-8'

So where do I tell my application to save the resultant XML file to be UTF-8? in the stylesheet or somewhere in VBA or both?

Thanks in advance.


Unknownscreenshots.doc
Postnext
(Deleted User) Subject: dodgy(non-ascii) characters causing confusion
Author: (Deleted User)
Date: 14 Feb 2007 01:02 PM
Richard,
it depend how your VBA application saves the file... if it relies on the MSXSL processor to actually write the file, it would be enough to add a <xsl:output encoding="utf-8"/> instruction right after the <xsl:stylesheet> node.
If your application is instead getting the result as a VBA string, you should investigate if the VBA file allows writing data in Unicode, instead of using the current locale.

Alberto
Postnext
Richard PottsSubject: dodgy(non-ascii) characters causing confusion
Author: Richard Potts
Date: 15 Feb 2007 06:48 AM
Thanks for the pointer Alberto.

In my VBA code I was creating the resultant file using "Writeline" i.e.
'
'Output to file
'
ExtractFile.WriteLine xslProc.output


Further Investigation on the web, for similar problems - found that ADO 'streams' seemed to be the answer: I've now changed it to:

'
' Open stream and set output code to UTF-8
'
Str.Charset = "UTF-8"
Str.Open
xslProc.output = Str

'
' Apply Transform
'
xslProc.transform

'
' Save stream to file
'
Str.SaveToFile sExtractFileName


So hope that helps someone else.

Not sure if it is possible or not… but as a thought, is there a potential enhancement to SS here? – e.g. to have a 'properties' for each file loaded into the XML editor so you can see what 'encoding' it was created with. Also is it possible to high-light the encoding issue to the user when you get the "? in diamonds" displayed for a xml file - How about maybe a SS feature 'convert this file from one encoding format to another encoding format' ???
Postnext
Tony LavinioSubject: dodgy(non-ascii) characters causing confusion
Author: Tony Lavinio
Date: 15 Feb 2007 09:40 AM
The XML standard says that if the encoding isn't in the first
line of the file, then it MUST be only one of a handful. The
rules are very specific, and anything else should be considered
broken and rejected by the parser.

See http://www.w3.org/TR/2006/REC-xml-20060816/#sec-guessing

The problem is that if the actual encoding of the file differs
from the stated encoding, then it is not always possible to determine
unambiguously what the encoding should have been. It is possible to
have a file that could be read successfully using several different
encodings and yielding several different results.

We do the best we can, but it is not deterministically possible to
tell in all cases, so the safest course is for Stylus Studio to
complain.
Posttop
Tony LavinioSubject: dodgy(non-ascii) characters causing confusion
Author: Tony Lavinio
Date: 15 Feb 2007 09:42 AM
... and there is a way inside of Stylus Studio to change the encoding,
but it assumes that the file in the editor is displayed properly.

Really, to fix a broken encoding you'd need a non-unicode-aware tool,
or more ideally you would get the source of the broken XML to not write
invalid XML.
   
Download A Free Trial of Stylus Studio 6 XML Professional Edition Today! Powered by Stylus Studio, the world's leading XML IDE for XML, XSLT, XQuery, XML Schema, DTD, XPath, WSDL, XHTML, SQL/XML, and XML Mapping!  
go

Log In Options

Site Map | Privacy Policy | Terms of Use | Trademarks
Stylus Scoop XML Newsletter:
W3C Member
Stylus Studio® and DataDirect XQuery ™are from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2016 All Rights Reserved.